{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://github.com/timeseriesAI/tsai/blob/main/tutorial_nbs/11_How_to_train_big_arrays_faster_with_tsai.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "created by Ignacio Oguiza - email: oguiza@timeseriesAI.co"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Purpose 😇"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> In this notebook, we'll demonstrate how you can efficiently train a larger-than-memory dataset using **tsai**.\n",
    "\n",
    "**Interested in speeding up training on larger-than-memory datasets?**\n",
    "\n",
    "If you want train your models fast you need to have a good / multiple good GPUs. No question about that. \n",
    "\n",
    "But even the best GPU will crawl if your batch creation process is slow. Only when the dataloaders create batches faster than the GPU consumes them, you'll achieve 100% GPU usage. And shorter training times. This is particularly important with larger-than-memory datasets.\n",
    "\n",
    "In this notebook we'll show you how to create really fast dataloaders to speed up your training.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Import libraries 📚"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # **************** UNCOMMENT AND RUN THIS CELL IF YOU NEED TO INSTALL/ UPGRADE TSAI ****************\n",
    "# stable = True # Set to True for latest pip version or False for main branch in GitHub\n",
    "# !pip install {\"tsai -U\" if stable else \"git+https://github.com/timeseriesAI/tsai.git\"} >> /dev/null"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "os             : Linux-5.4.104+-x86_64-with-Ubuntu-18.04-bionic\n",
      "python         : 3.7.11\n",
      "tsai           : 0.2.23\n",
      "fastai         : 2.5.2\n",
      "fastcore       : 1.3.26\n",
      "zarr           : 2.9.5\n",
      "torch          : 1.9.0+cu102\n",
      "n_cpus         : 2\n",
      "device         : cuda (Tesla T4)\n"
     ]
    }
   ],
   "source": [
    "from tsai.all import *\n",
    "import zarr\n",
    "my_setup(zarr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Zarr arrays ☦️"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition to tsai, we'll make also need a library called [zarr](https://zarr.readthedocs.io). \n",
    "\n",
    "\n",
    "Zarr is a format for the storage of chunked, compressed, N-dimensional arrays. These documents describe the Zarr format and its Python implementation.\n",
    "\n",
    "**Highlights**\n",
    "\n",
    "- Create N-dimensional arrays with any NumPy dtype.\n",
    "- Chunk arrays along any dimension.\n",
    "- Compress and/or filter chunks using any NumCodecs codec.\n",
    "- Store arrays in memory, on disk, inside a Zip file, on S3, …\n",
    "- Read an array concurrently from multiple threads or processes.\n",
    "- Write to an array concurrently from multiple threads or processes.\n",
    "- Organize arrays into hierarchies via groups."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Want to know more about zarr arrays?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are going to briefly understand how we can use zarr arrays. Let's prepare a relatively small dataset that fits in memory. \n",
    "\n",
    "Note: For a more in-depth understanding of zarr arrays you can visit this [tutorial](https://zarr.readthedocs.io/en/stable/tutorial.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "path = Path('data')\n",
    "if not os.path.exists(path): os.makedirs(path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr = np.random.rand(10_000, 10, 100)\n",
    "np.save(path/'arr.npy', arr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We know that one of the data formats we can use for large datasets are memory-mapped arrays (np.memmap). They behave like normal arrays, except that they remain on disk. When you index them, only those indices are loaded in memory. That's great because it's what we need to create batches in tsai."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10000, 10, 100)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "memmap_arr = np.load(path/'arr.npy', mmap_mode='r+')\n",
    "memmap_arr.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create an on-disk zarr array & load data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For large datasets, we need to create an zarr array on disk. The array will initially be empty, with all values set to 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<zarr.core.Array (10000, 10, 100) float64>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "zarr_arr = zarr.open(path/'zarr.zarr', mode='w', shape=arr.shape, dtype=arr.dtype)\n",
    "zarr_arr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "They are array-like objects, like memmap arrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(True, True)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hasattr(memmap_arr, '__array__'), hasattr(zarr_arr, '__array__')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Although they are not np.ndarrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(True, False)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "isinstance(memmap_arr, np.ndarray), isinstance(zarr_arr, np.ndarray)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To load data into the array we can: \n",
    "\n",
    "- assign it directly if the data fits in memory or if data is stored in a memmap array on disk: \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "zarr_arr[:] = memmap_arr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- we can split data in chunks that fit memory. We can load data from an array, a dataframe, etc. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "chunksize = 1_000\n",
    "\n",
    "for i in range(len(memmap_arr) // chunksize + 1):\n",
    "    zarr_arr[i * chunksize : (i + 1) * chunksize] = memmap_arr[i * chunksize : (i + 1) * chunksize]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is a function in `tsai` that helps you assign data in chunks in place:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "assign_in_chunks(zarr_arr, memmap_arr, chunksize=10_000, inplace=True, verbose=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once you load data, if you want to ensure your data is never modified, you may load the zarr array in mode 'r' (read only). This process is fast.\n",
    "\n",
    "Otherwise you can continue working with your array in mode 'w'.\n",
    "\n",
    "Note: even if zerr.open seems to indicate the opposite, there's no need to ever close zarr arrays. You can just delete as a regular array, and recursively remove if from its directory (if on disk)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "zarr_arr = zarr.open(path/'zarr.zarr', mode='r')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How to slice a zarr array?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now create some indices to index the arrays. \n",
    "\n",
    "Indexing is used to obtain individual elements from an array. In tsai, it's used to create batches of data by the dataloader. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "idxs = random_choice(len(arr), 64, False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((64, 10, 100), (64, 10, 100))"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr[idxs].shape, memmap_arr[idxs].shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When memmap arrays are index, the index elements become a regular np.array. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(numpy.ndarray, numpy.ndarray)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(arr[idxs]), type(memmap_arr[idxs])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Zarr arrays are unique in the way they need to be indexed.\n",
    "\n",
    "If we try to apply indices in the same way as with memmap arrays, you'll get an error message.\n",
    "\n",
    "Zarr arrays have an attribute called 'oindex' that is used to index the array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hasattr(zarr_arr, 'oindex')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((64, 10, 100), numpy.ndarray)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "zarr_arr.oindex[idxs].shape, type(zarr_arr.oindex[idxs])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(True, True)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.array_equal(arr[idxs], memmap_arr[idxs]), np.array_equal(arr[idxs], zarr_arr.oindex[idxs])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "tsai supports many types of array-like objects, like numpy arrays, memmap arrays, zarr arrays, xarrays, dask arrays, and even lists or L objects. \n",
    "\n",
    "That means you can use any of those types as data inputs. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# In-memory datasets 🧠"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's compare the performance of the 3 types of arrays we have created (np.array, np.memmap, zarr). They all fit in memory, although for comparison, np.memmap and zarr data are on disk. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1000 loops, best of 5: 1.33 ms per loop\n"
     ]
    }
   ],
   "source": [
    "%timeit arr[random_choice(len(arr), 512, False)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1000 loops, best of 5: 1.33 ms per loop\n"
     ]
    }
   ],
   "source": [
    "%timeit memmap_arr[random_choice(len(arr), 512, False)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we'll create a zarr array. Since the array fits in memory, we'll set chunks to False, meaning no chunking will be applied. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(<zarr.core.Array (10000, 10, 100) float64>, (10000, 10, 100))"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "zarr_arr = zarr.open(path/'zarr.zarr', mode='w', shape=arr.shape, dtype=arr.dtype, chunks=False) \n",
    "zarr_arr[:] = arr \n",
    "zarr_arr, zarr_arr.chunks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10 loops, best of 5: 63 ms per loop\n"
     ]
    }
   ],
   "source": [
    "%timeit zarr_arr.oindex[random_choice(len(arr), 512, False)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "⚠️ In order to improve performance, it's beneficial for zarr arrays to sort the indices. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10 loops, best of 5: 60.6 ms per loop\n"
     ]
    }
   ],
   "source": [
    "%timeit zarr_arr.oindex[np.sort(random_choice(len(arr), 512, False))]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have learned 2 things: \n",
    "\n",
    "- For datasets that fit in memory, both numpy arrays and zarr arrays are very fast. Memmap arrays are slower.\n",
    "\n",
    "- Chunks have a great impact on zarr arrays performance. Always choose chunks=False for datasets that fit in memory. \n",
    "\n",
    "⚠️ chunks=False means a single data containing all data in the dataset. We can only do this when data fits in memory. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Larger-than-memory datasets 🤯"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are going to create a larger-than_memory file with **dummy data**. \n",
    "\n",
    "Colab Pro (standard) has 13GB of RAM. However, the X_large file has 37GB. So we'll need to manage everything on disk.\n",
    "\n",
    "Data creation will take around 10 min.\n",
    "\n",
    "\n",
    "⚠️ Remember to delete this file when you finish!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "path = Path('data')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uncomment to create the data\n",
    "# X_large = create_array((1_000_000, 10, 1000), fname='X_large', path='data', mode='r+')\n",
    "# del X_large"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1000000, 10, 1000)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_large = np.load(path/'X_large.npy', mmap_mode='r+')\n",
    "X_large.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "memmap(['b', 'b', 'a', ..., 'a', 'c', 'c'], dtype='<U1')"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_large = np.array(['a', 'b', 'c'])[np.random.randint(0, 3, len(X_large))]\n",
    "np.save(path/'y_large.npy', y_large)\n",
    "del y_large\n",
    "y_large = np.load(path/'y_large.npy', mmap_mode='r+')\n",
    "y_large"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABAYAAABXCAYAAACXx0eYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAQ+klEQVR4nO3de5DV5X3H8c9nWeQyIMmyK7fVkKIiuLKQJRCtAUIyVG3QMQS8gMZUtOO0aVpSJJ1k1KK1aXrRaTWtlxBITShGjYPWlKQtgpdIBZL1goCEkEBkw+IuEiQb9rDf/nF+29lu9nJ2PZdlz/s143jOeZ7zfb5Hnzmzv+95nufniBAAAAAAAChOJYVOAAAAAAAAFA6FAQAAAAAAihiFAQAAAAAAihiFAQAAAAAAihiFAQAAAAAAihiFAQAAAAAAihiFAQDAKcv2s7aXJo8X2/7+e4g13nbYLk2ef8/2Z7KU50dt72rzfJ/tT2QjdhLvddtzshUPAAAUFwoDAICCsn2x7Rdtv2O7wfYLtj/c0zgR8a2ImNcmbtg+u7d5RcSlEbGmu36ZjBMRz0XExN7m0m681bbvahf//Ih4NhvxAQBA8SktdAIAgOJl+3RJT0u6RdKjkk6T9FFJvylkXtlkuzQiUoXOAwAAoDOsGAAAFNK5khQRayPiZET8OiK+HxGvSJLtG5IVBPclKwp22v54R4GSvs8njzcnL9faPmb7qg76D7D9d7YP294r6ffbtbfdpnC27U1JDodtr+tsHNtzbB+wvcJ2naRvtL7WLoUP295hu9H2N2wPbv852uQSSQ43S1os6dZkvKeS9v/bmmB7kO17bb+V/HOv7UFJW2tuX7B9yPZB25/t9v8SAADo1ygMAAAKabekk7bX2L7U9vs76DNT0k8klUu6XdITtsu6ChoRs5KH1RExLCLWddDtJkmflDRN0nRJn+4i5J2Svi/p/ZIqJf1TN+OMllQm6QOSbu4k5mJJvydpgtIFki939ZmS8R6U9C1JX03Gm99Bty9J+oikqZKqJc1oF3u0pBGSxkm6UdL9nfx3BwAARYLCAACgYCLiqKSLJYWkhyTV215ve1Sbbock3RsRzcmF9y61+3W/lxYlcfdHRIOkv+6ib7PSF/ljI6IpIp7voq8ktUi6PSJ+ExG/7qTPfW3G/itJ1/T0A3RisaSVEXEoIuol/aWk69q0NyftzRHxjKRjkrJy/gEAADg1URgAABRURLwRETdERKWkKkljJd3bpssvIiLaPP9Z0ue9Gitpf7u4nblVkiX9T3IHgD/oJnZ9RDR106f92Nn4TEritP0s7WO/3e7Mg+OShmVpbAAAcAqiMAAA6DMiYqek1UoXCFqNs+02z8+S9FYWhjso6cx2cTvLqy4iboqIsZL+UNLXurkTQXTR1qr92K2f6V1JQ1sbbI/uYey3lF7d0FFsAACA30JhAABQMLbPSw7Cq0yen6n0kvqX2nQ7Q9Kf2B5oe6GkSZKeySD8LyX9ThftjyZxK5M99l/sIs+FrTlKalT64rwlw3E680fJ2GVKnwvQej5BraTzbU9NDiS8o937uhtvraQv266wXS7pNkmP9CI/AABQJCgMAAAK6VdKHy64xfa7ShcEXpP0hTZ9tkg6R9Jhpffifzoi3s4g9h2S1tg+YntRB+0PSdqg9IX4dklPdBHrw0mOxyStl/T5iNib4Tid+bbSBxruVfpwxbskKSJ2S1op6T8lvSmp/XkGX5c0ORnvyQ7i3iVpq6RXJL2afLa7epAXAAAoMv7/2zYBAOg7bN8gaWlEXFzoXAAAAPorVgwAAAAAAFDEKAwAAAAAAFDE2EoAAAAAAEARY8UAAAAAAABFjMIAAAAAAABFrDQXQe3ykMbnIjQAAABwSho66Y1CpwBkxfE3jh+OiIpC54HsyUlhIF0U2Jqb0AAAAMAp6LxHagqdApAV22u2/6zQOSC72EoAAAAAAEARozAAAAAAAEARozAAAAAAAEARy9EZAwAAAAAA9F3btm07o7S09GFJVerfP5q3SHotlUotrampOdRRBwoDAAAAAICiU1pa+vDo0aMnVVRUNJaUlESh88mVlpYW19fXT66rq3tY0uUd9enPVREAAAAAADpTVVFRcbQ/FwUkqaSkJCoqKt5RemVEx33ymA8AAAAAAH1FSX8vCrRKPmen1/9sJQAAAAAAIM/q6uoGzJkzZ6IkHT58eGBJSUmUlZWlJOnHP/7xG4MHD+60aLF58+ahq1atGrl69er92cil28KA7VWSPinpUER0uvQAAAAAAIBTla2abMaL0Lau2kePHn1y586dOyRp2bJlY4cNG3Zy5cqVv2xtb25u1sCBAzt876xZs47PmjXreLZyzWQrwWpJl2RrQAAAAAAA8NsWLFgw/tprrz1rypQp591yyy2VGzduHDp16tTzJk2aNHnatGnn1dbWDpKkp59+evjHPvaxs6V0UWHhwoXjZ8yYMbGysvKCu+6664yejtvtioGI2Gx7fE8DAwAAAACAnjl48OBp27dv31laWqqGhoaSl19+eefAgQP15JNPDr/11lsrN2zY8JP279mzZ8/gF198cdeRI0cGTJo0qWr58uX1gwYNyvj8hKydMWD7Zkk3p5+dla2wAAAAAAAUjU996lONpaXpS/WGhoYBV1111Qf37ds32HY0Nze7o/fMmzfvyJAhQ2LIkCGpsrKy5gMHDpROmDChOdMxs3ZXgoh4MCKmR8R0qSJbYQEAAAAAKBrDhg1raX28YsWKcbNnz/7Vm2+++fpTTz2158SJEx1ew7ddHTBgwAClUqkOCwid4XaFAAAAAAD0QUePHh1QWVl5QpIeeOCB8lyNQ2EAAAAAAIA+aMWKFXV33HFH5aRJkyanUqmcjeOIrs8jsL1W0hxJ5ZJ+Ken2iPh61++ZHtLWbOUIAAAAnPI+tC2rd0IDCmZ7zfZt6S3kp7ba2tp91dXVhwudR77U1taWV1dXj++oLZO7ElyT9YwAAAAAAECfwFYCAAAAAACKGIUBAAAAAACKGIUBAAAAAACKGIUBAAAAAACKGIUBAAAAAACKGIUBAAAAAADybObMmec+/vjjp7d9beXKlWcsXrz4rI76z5gxY+LmzZuHStLs2bPPPnz48ID2fZYtWzb2tttuG9XTXLq9XSEAAAAAAP1dzfaammzG2/ahbdu6al+4cGHD2rVryxYsWHC09bXHH3+87Ctf+cqB7mJv2rRpTzZybJWTwkBNjbR1ay4iAwAAAKeqLq8RgFOG5UKn0C9cd911jXffffe4pqYmDx48OHbt2nXaoUOHBj7yyCNly5cvP7Opqalk/vz5jffcc89b7d87bty4C7Zu3frGmDFjUitWrBi9bt268pEjRzaPHTv2xLRp0473NJecFAa2bdt2zPauXMQG8qhc0uFCJwFkAXMZ/QHzGP0Fcxn9wcRCJ9AfjBo16mR1dfW7jz322IglS5YcWbNmTdn8+fMb77zzzoOjRo06mUqldNFFF03csmXLkJkzZ/66oxjPPffc0O9+97tlr7766o7m5mZNnTp1cp8pDEjaFRHTcxQbyAvbW5nH6A+Yy+gPmMfoL5jL6A9ssz48SxYtWtSwbt269y9ZsuTIE088UfbQQw/tW7NmTdnq1avLU6mU6+vrB9bW1g7urDCwcePGYZdddtmR4cOHt0jSvHnzjvQmDw4fBAAAAACgAK699tojL7zwwunPP//80KamppKKiorUfffdN2rTpk27d+/evWPu3LnvNDU15fy6ncIAAAAAAAAFMGLEiJYLL7zwV0uXLh1/5ZVXNjQ2Ng4YMmRIS1lZ2cn9+/eXPvvssyO6ev/cuXOPPfPMM+87duyYGxsbS37wgx+8rzd55GorwYM5igvkE/MY/QVzGf0B8xj9BXMZ/QHzOIuuvvrqhuuvv37C2rVr906bNq2pqqrq+IQJE6rGjBlzoqam5lhX77344ouPX3nllQ1VVVXnjxw5snnKlCnv9iYHR0TvsgcAAAAA4BRVW1u7r7q6umgOA62trS2vrq4e31EbWwkAAAAAAChivS4M2L7E9i7be2x/sYP2QbbXJe1bbI9/L4kCuZLBXF5me4ftV2z/l+0PFCJPoCvdzeM2/RbYDtuciI0+KZO5bHtR8r38uu1v5ztHIBMZ/H1xlu2Ntn+U/I1xWSHyBLpie5XtQ7Zf66Tdtv8xmeev2P5QvnNEdvSqMGB7gKT7JV0qabKka2xPbtftRkmNEXG2pHsk/c17SRTIhQzn8o8kTY+IKZIek/TV/GYJdC3DeSzbwyV9XtKW/GYIZCaTuWz7HEl/Iel3I+J8SX+a90SBbmT4vfxlSY9GxDRJV0v6Wn6zBDKyWtIlXbRfKumc5J+bJf1zHnJCDvR2xcAMSXsiYm9EnJD0b5KuaNfnCklrksePSfq4bfdyPCBXup3LEbExIo4nT1+SVJnnHIHuZPKdLEl3Kl2kbcpnckAPZDKXb5J0f0Q0SlJEHMpzjkAmMpnLIen05PEISW/lMT8gIxGxWVJDF12ukPTNSHtJ0vtsj8lPdlnR0tLSUhTXqMnnbOmsvbeFgXGS9rd5fiB5rcM+EZGS9I6kkb0cD8iVTOZyWzdK+l5OMwJ6rtt5nCztOzMi/j2fiQE9lMl38rmSzrX9gu2XbHf1SxZQKJnM5TskLbF9QNIzkj6Xn9SArOrp39J9zWv19fUj+ntxoKWlxfX19SMkdbglRMrd7QqBfsf2EknTJc0udC5AT9gukfQPkm4ocCpANpQqvWR1jtIruDbbviAijhQ0K6DnrpG0OiL+3vaFkv7VdlVEdPqLHoDsSqVSS+vq6h6uq6urUv8+mL9F0mupVGppZx16Wxj4haQz2zyvTF7rqM8B26VKL5F6u5fjAbmSyVyW7U9I+pKk2RHxmzzlBmSqu3k8XFKVpGeTHV2jJa23fXlEbM1blkD3MvlOPiBpS0Q0S/qp7d1KFwpezk+KQEYymcs3Ktm7HRE/tD1YUrkktsfgVJLR39J9VU1NzSFJlxc6j76gt1WRlyWdY/uDtk9T+sCU9e36rJf0meTxpyX9d0REL8cDcqXbuWx7mqQHJF3OXlb0UV3O44h4JyLKI2J8RIxX+qwMigLoizL5++JJpVcLyHa50lsL9uYzSSADmczln0v6uCTZniRpsKT6vGYJvHfrJV2f3J3gI5LeiYiDhU4KPderFQMRkbL9x5I2SBogaVVEvG57paStEbFe0teVXhK1R+kDK67OVtJAtmQ4l/9W0jBJ30l+bf15RFBZRJ+R4TwG+rwM5/IGSfNs75B0UtLyiGBFIvqUDOfyFyQ9ZPvPlD6I8AZ+RENfY3ut0sXY8uQ8jNslDZSkiPgXpc/HuEzSHknHJX22MJnivTLfPwAAAAAAFK/+fMACAAAAAADoBoUBAAAAAACKGIUBAAAAAACKGIUBAAAAAACKGIUBAAAAACgStlfZPmT7tQz7L7K9w/brtr+d6/xQGNyVAAAAAACKhO1Zko5J+mZEVHXT9xxJj0qaGxGNts+IiEP5yBP5xYoBAAAAACgSEbFZUkPb12xPsP0ftrfZfs72eUnTTZLuj4jG5L0UBfopCgMAAAAAUNwelPS5iKiR9OeSvpa8fq6kc22/YPsl25cULEPkVGmhEwAAAAAAFIbtYZIukvQd260vD0r+XSrpHElzJFVK2mz7gog4ku88kVsUBgAAAACgeJVIOhIRUztoOyBpS0Q0S/qp7d1KFwpezmeCyD22EgAAAABAkYqIo0pf9C+UJKdVJ81PKr1aQLbLld5asLcQeSK3KAwAAAAAQJGwvVbSDyVNtH3A9o2SFku60XatpNclXZF03yDpbds7JG2UtDwi3i5E3sgtblcIAAAAAEARY8UAAAAAAABFjMIAAAAAAABFjMIAAAAAAABFjMIAAAAAAABFjMIAAAAAAABFjMIAAAAAAABFjMIAAAAAAABFjMIAAAAAAABF7H8BUP9a40NShegAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1152x36 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "splits = TimeSplitter()(y_large)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see what's the performance of memmap arrays when they are indexed (simulating a batch creation process): "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<TimeitResult : 1 loop, best of 5: 1.72 s per loop>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np_res = %timeit -qo X_large[random_choice(len(X_large), 512, False)]\n",
    "np_res"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now create a zarr array on disk with the same shape. In this case, since data doesn't fit in memory, we are going to use chunks to be able to improve performance. The recommended approach is to set chunks != -1 in the dimension we are going to use (in our case the first dimension). That's why we set chunks=(1, -1,-1),\n",
    "For an in-depth review on chunks you can read [this](https://docs.dask.org/en/latest/array-chunks.html).\n",
    "\n",
    "When creating a zarr array we need to use mode='a' (read if exist, otherwise write) or 'w' (always write or overwrite). There's no need to ever close the array. \n",
    "\n",
    "When an array's been created, you can always read it again bu setting mode='r'.\n",
    "\n",
    "This step will also take about 10 min.\n",
    "\n",
    "⚠️ Note that chunks stands for “chunk shape” rather than “number of chunks”, so specifying chunks=1 means that you will have many chunks, each with exactly one element."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# X_large_zarr = zarr.open(path/'X_large.zarr', mode='w', shape=X_large.shape, dtype=X_large.dtype, chunks=(1, -1, -1)) # chunks=(1, -1, -1) == (1, None, None)\n",
    "# X_large_zarr[:] = X_large"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<zarr.core.Array (1000000, 10, 1000) float32>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_large_zarr = zarr.open(path/'X_large.zarr', mode='r')\n",
    "X_large_zarr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.31s which is 24.0% less\n"
     ]
    }
   ],
   "source": [
    "zarr_res = %timeit -qo X_large_zarr.oindex[random_choice(len(X_large), 512, False)]\n",
    "print(f'{zarr_res.best:.2f}s which is {1 - zarr_res.best/np_res.best:.1%} less')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Excellent!!! 🙃 I'm starting to really like zarr arrays!!!\n",
    "\n",
    "One more idea. Zarr arrays are a bit faster when the indices are sorted. Let's try it: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.19s which is 30.8% less\n"
     ]
    }
   ],
   "source": [
    "zarr_sorted_res = %timeit -qo X_large_zarr.oindex[np.sort(random_choice(len(X_large), 512, False))]\n",
    "print(f'{zarr_sorted_res.best:.2f}s which is {1 - zarr_sorted_res.best/np_res.best:.1%} less')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "This is an excellent performance for a large, on-disk dataset like this. Indexing is around 30% faster than memmap arrays!!! 🚀\n",
    "\n",
    "Zarr arrays are very useful when dealing with larger-then-memory datasets.\n",
    "\n",
    "There are some important concepts to unsertand why zarr arrays are faster to index: \n",
    "\n",
    "- Chunks are parts of the array that are loaded in memory at the same time. This makes indexing more efficient. \n",
    "\n",
    "- Zarr arrays are compressed. This also improves the loading of data in memory. \n",
    "\n",
    "- Zarr arrays can be concurrently read and written by multiple threads or processes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Key learnings: "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- For datasets that fit in memory, use numpy arrays (or np.memmap arrays). \n",
    "\n",
    "- For larger than memory datasets, use zarr arrays. \n",
    "\n",
    "- Set up chunks=(1, -1, -1)\n",
    "\n",
    "- Sort the indices before indexing the array. \n",
    "\n",
    "\n",
    "The tsai library applies all these learnings to achieve the fastest possible performance for larger-than-memory datasets."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training  🏃🏽‍♀️"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABAYAAABXCAYAAACXx0eYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAQ+klEQVR4nO3de5DV5X3H8c9nWeQyIMmyK7fVkKIiuLKQJRCtAUIyVG3QMQS8gMZUtOO0aVpSJJ1k1KK1aXrRaTWtlxBITShGjYPWlKQtgpdIBZL1goCEkEBkw+IuEiQb9rDf/nF+29lu9nJ2PZdlz/s143jOeZ7zfb5Hnzmzv+95nufniBAAAAAAAChOJYVOAAAAAAAAFA6FAQAAAAAAihiFAQAAAAAAihiFAQAAAAAAihiFAQAAAAAAihiFAQAAAAAAihiFAQDAKcv2s7aXJo8X2/7+e4g13nbYLk2ef8/2Z7KU50dt72rzfJ/tT2QjdhLvddtzshUPAAAUFwoDAICCsn2x7Rdtv2O7wfYLtj/c0zgR8a2ImNcmbtg+u7d5RcSlEbGmu36ZjBMRz0XExN7m0m681bbvahf//Ih4NhvxAQBA8SktdAIAgOJl+3RJT0u6RdKjkk6T9FFJvylkXtlkuzQiUoXOAwAAoDOsGAAAFNK5khQRayPiZET8OiK+HxGvSJLtG5IVBPclKwp22v54R4GSvs8njzcnL9faPmb7qg76D7D9d7YP294r6ffbtbfdpnC27U1JDodtr+tsHNtzbB+wvcJ2naRvtL7WLoUP295hu9H2N2wPbv852uQSSQ43S1os6dZkvKeS9v/bmmB7kO17bb+V/HOv7UFJW2tuX7B9yPZB25/t9v8SAADo1ygMAAAKabekk7bX2L7U9vs76DNT0k8klUu6XdITtsu6ChoRs5KH1RExLCLWddDtJkmflDRN0nRJn+4i5J2Svi/p/ZIqJf1TN+OMllQm6QOSbu4k5mJJvydpgtIFki939ZmS8R6U9C1JX03Gm99Bty9J+oikqZKqJc1oF3u0pBGSxkm6UdL9nfx3BwAARYLCAACgYCLiqKSLJYWkhyTV215ve1Sbbock3RsRzcmF9y61+3W/lxYlcfdHRIOkv+6ib7PSF/ljI6IpIp7voq8ktUi6PSJ+ExG/7qTPfW3G/itJ1/T0A3RisaSVEXEoIuol/aWk69q0NyftzRHxjKRjkrJy/gEAADg1URgAABRURLwRETdERKWkKkljJd3bpssvIiLaPP9Z0ue9Gitpf7u4nblVkiX9T3IHgD/oJnZ9RDR106f92Nn4TEritP0s7WO/3e7Mg+OShmVpbAAAcAqiMAAA6DMiYqek1UoXCFqNs+02z8+S9FYWhjso6cx2cTvLqy4iboqIsZL+UNLXurkTQXTR1qr92K2f6V1JQ1sbbI/uYey3lF7d0FFsAACA30JhAABQMLbPSw7Cq0yen6n0kvqX2nQ7Q9Kf2B5oe6GkSZKeySD8LyX9ThftjyZxK5M99l/sIs+FrTlKalT64rwlw3E680fJ2GVKnwvQej5BraTzbU9NDiS8o937uhtvraQv266wXS7pNkmP9CI/AABQJCgMAAAK6VdKHy64xfa7ShcEXpP0hTZ9tkg6R9Jhpffifzoi3s4g9h2S1tg+YntRB+0PSdqg9IX4dklPdBHrw0mOxyStl/T5iNib4Tid+bbSBxruVfpwxbskKSJ2S1op6T8lvSmp/XkGX5c0ORnvyQ7i3iVpq6RXJL2afLa7epAXAAAoMv7/2zYBAOg7bN8gaWlEXFzoXAAAAPorVgwAAAAAAFDEKAwAAAAAAFDE2EoAAAAAAEARY8UAAAAAAABFjMIAAAAAAABFrDQXQe3ykMbnIjQAAABwSho66Y1CpwBkxfE3jh+OiIpC54HsyUlhIF0U2Jqb0AAAAMAp6LxHagqdApAV22u2/6zQOSC72EoAAAAAAEARozAAAAAAAEARozAAAAAAAEARy9EZAwAAAAAA9F3btm07o7S09GFJVerfP5q3SHotlUotrampOdRRBwoDAAAAAICiU1pa+vDo0aMnVVRUNJaUlESh88mVlpYW19fXT66rq3tY0uUd9enPVREAAAAAADpTVVFRcbQ/FwUkqaSkJCoqKt5RemVEx33ymA8AAAAAAH1FSX8vCrRKPmen1/9sJQAAAAAAIM/q6uoGzJkzZ6IkHT58eGBJSUmUlZWlJOnHP/7xG4MHD+60aLF58+ahq1atGrl69er92cil28KA7VWSPinpUER0uvQAAAAAAIBTla2abMaL0Lau2kePHn1y586dOyRp2bJlY4cNG3Zy5cqVv2xtb25u1sCBAzt876xZs47PmjXreLZyzWQrwWpJl2RrQAAAAAAA8NsWLFgw/tprrz1rypQp591yyy2VGzduHDp16tTzJk2aNHnatGnn1dbWDpKkp59+evjHPvaxs6V0UWHhwoXjZ8yYMbGysvKCu+6664yejtvtioGI2Gx7fE8DAwAAAACAnjl48OBp27dv31laWqqGhoaSl19+eefAgQP15JNPDr/11lsrN2zY8JP279mzZ8/gF198cdeRI0cGTJo0qWr58uX1gwYNyvj8hKydMWD7Zkk3p5+dla2wAAAAAAAUjU996lONpaXpS/WGhoYBV1111Qf37ds32HY0Nze7o/fMmzfvyJAhQ2LIkCGpsrKy5gMHDpROmDChOdMxs3ZXgoh4MCKmR8R0qSJbYQEAAAAAKBrDhg1raX28YsWKcbNnz/7Vm2+++fpTTz2158SJEx1ew7ddHTBgwAClUqkOCwid4XaFAAAAAAD0QUePHh1QWVl5QpIeeOCB8lyNQ2EAAAAAAIA+aMWKFXV33HFH5aRJkyanUqmcjeOIrs8jsL1W0hxJ5ZJ+Ken2iPh61++ZHtLWbOUIAAAAnPI+tC2rd0IDCmZ7zfZt6S3kp7ba2tp91dXVhwudR77U1taWV1dXj++oLZO7ElyT9YwAAAAAAECfwFYCAAAAAACKGIUBAAAAAACKGIUBAAAAAACKGIUBAAAAAACKGIUBAAAAAACKGIUBAAAAAADybObMmec+/vjjp7d9beXKlWcsXrz4rI76z5gxY+LmzZuHStLs2bPPPnz48ID2fZYtWzb2tttuG9XTXLq9XSEAAAAAAP1dzfaammzG2/ahbdu6al+4cGHD2rVryxYsWHC09bXHH3+87Ctf+cqB7mJv2rRpTzZybJWTwkBNjbR1ay4iAwAAAKeqLq8RgFOG5UKn0C9cd911jXffffe4pqYmDx48OHbt2nXaoUOHBj7yyCNly5cvP7Opqalk/vz5jffcc89b7d87bty4C7Zu3frGmDFjUitWrBi9bt268pEjRzaPHTv2xLRp0473NJecFAa2bdt2zPauXMQG8qhc0uFCJwFkAXMZ/QHzGP0Fcxn9wcRCJ9AfjBo16mR1dfW7jz322IglS5YcWbNmTdn8+fMb77zzzoOjRo06mUqldNFFF03csmXLkJkzZ/66oxjPPffc0O9+97tlr7766o7m5mZNnTp1cp8pDEjaFRHTcxQbyAvbW5nH6A+Yy+gPmMfoL5jL6A9ssz48SxYtWtSwbt269y9ZsuTIE088UfbQQw/tW7NmTdnq1avLU6mU6+vrB9bW1g7urDCwcePGYZdddtmR4cOHt0jSvHnzjvQmDw4fBAAAAACgAK699tojL7zwwunPP//80KamppKKiorUfffdN2rTpk27d+/evWPu3LnvNDU15fy6ncIAAAAAAAAFMGLEiJYLL7zwV0uXLh1/5ZVXNjQ2Ng4YMmRIS1lZ2cn9+/eXPvvssyO6ev/cuXOPPfPMM+87duyYGxsbS37wgx+8rzd55GorwYM5igvkE/MY/QVzGf0B8xj9BXMZ/QHzOIuuvvrqhuuvv37C2rVr906bNq2pqqrq+IQJE6rGjBlzoqam5lhX77344ouPX3nllQ1VVVXnjxw5snnKlCnv9iYHR0TvsgcAAAAA4BRVW1u7r7q6umgOA62trS2vrq4e31EbWwkAAAAAAChivS4M2L7E9i7be2x/sYP2QbbXJe1bbI9/L4kCuZLBXF5me4ftV2z/l+0PFCJPoCvdzeM2/RbYDtuciI0+KZO5bHtR8r38uu1v5ztHIBMZ/H1xlu2Ntn+U/I1xWSHyBLpie5XtQ7Zf66Tdtv8xmeev2P5QvnNEdvSqMGB7gKT7JV0qabKka2xPbtftRkmNEXG2pHsk/c17SRTIhQzn8o8kTY+IKZIek/TV/GYJdC3DeSzbwyV9XtKW/GYIZCaTuWz7HEl/Iel3I+J8SX+a90SBbmT4vfxlSY9GxDRJV0v6Wn6zBDKyWtIlXbRfKumc5J+bJf1zHnJCDvR2xcAMSXsiYm9EnJD0b5KuaNfnCklrksePSfq4bfdyPCBXup3LEbExIo4nT1+SVJnnHIHuZPKdLEl3Kl2kbcpnckAPZDKXb5J0f0Q0SlJEHMpzjkAmMpnLIen05PEISW/lMT8gIxGxWVJDF12ukPTNSHtJ0vtsj8lPdlnR0tLSUhTXqMnnbOmsvbeFgXGS9rd5fiB5rcM+EZGS9I6kkb0cD8iVTOZyWzdK+l5OMwJ6rtt5nCztOzMi/j2fiQE9lMl38rmSzrX9gu2XbHf1SxZQKJnM5TskLbF9QNIzkj6Xn9SArOrp39J9zWv19fUj+ntxoKWlxfX19SMkdbglRMrd7QqBfsf2EknTJc0udC5AT9gukfQPkm4ocCpANpQqvWR1jtIruDbbviAijhQ0K6DnrpG0OiL+3vaFkv7VdlVEdPqLHoDsSqVSS+vq6h6uq6urUv8+mL9F0mupVGppZx16Wxj4haQz2zyvTF7rqM8B26VKL5F6u5fjAbmSyVyW7U9I+pKk2RHxmzzlBmSqu3k8XFKVpGeTHV2jJa23fXlEbM1blkD3MvlOPiBpS0Q0S/qp7d1KFwpezk+KQEYymcs3Ktm7HRE/tD1YUrkktsfgVJLR39J9VU1NzSFJlxc6j76gt1WRlyWdY/uDtk9T+sCU9e36rJf0meTxpyX9d0REL8cDcqXbuWx7mqQHJF3OXlb0UV3O44h4JyLKI2J8RIxX+qwMigLoizL5++JJpVcLyHa50lsL9uYzSSADmczln0v6uCTZniRpsKT6vGYJvHfrJV2f3J3gI5LeiYiDhU4KPderFQMRkbL9x5I2SBogaVVEvG57paStEbFe0teVXhK1R+kDK67OVtJAtmQ4l/9W0jBJ30l+bf15RFBZRJ+R4TwG+rwM5/IGSfNs75B0UtLyiGBFIvqUDOfyFyQ9ZPvPlD6I8AZ+RENfY3ut0sXY8uQ8jNslDZSkiPgXpc/HuEzSHknHJX22MJnivTLfPwAAAAAAFK/+fMACAAAAAADoBoUBAAAAAACKGIUBAAAAAACKGIUBAAAAAACKGIUBAAAAACgStlfZPmT7tQz7L7K9w/brtr+d6/xQGNyVAAAAAACKhO1Zko5J+mZEVHXT9xxJj0qaGxGNts+IiEP5yBP5xYoBAAAAACgSEbFZUkPb12xPsP0ftrfZfs72eUnTTZLuj4jG5L0UBfopCgMAAAAAUNwelPS5iKiR9OeSvpa8fq6kc22/YPsl25cULEPkVGmhEwAAAAAAFIbtYZIukvQd260vD0r+XSrpHElzJFVK2mz7gog4ku88kVsUBgAAAACgeJVIOhIRUztoOyBpS0Q0S/qp7d1KFwpezmeCyD22EgAAAABAkYqIo0pf9C+UJKdVJ81PKr1aQLbLld5asLcQeSK3KAwAAAAAQJGwvVbSDyVNtH3A9o2SFku60XatpNclXZF03yDpbds7JG2UtDwi3i5E3sgtblcIAAAAAEARY8UAAAAAAABFjMIAAAAAAABFjMIAAAAAAABFjMIAAAAAAABFjMIAAAAAAABFjMIAAAAAAABFjMIAAAAAAABFjMIAAAAAAABF7H8BUP9a40NShegAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1152x36 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "X_large_zarr = zarr.open(path/'X_large.zarr', mode='r') # mode='r' because we are reasing the zarr array we previously created\n",
    "y_large_zarr = zarr.open(path/'y_large.zarr', mode='w', shape=y_large.shape, dtype=y_large.dtype, chunks=False) # y data is small and don't need to be chunked\n",
    "y_large_zarr[:] = y_large\n",
    "splits = TimeSplitter()(y_large)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When creating the dataloaders, there are 2 important things to remember:\n",
    "\n",
    "1. If you are dealing with a classification task and need to transform the lavels, use TSClassification. It's vectorized version of Categorize that run much faster.\n",
    "\n",
    "2. Set inplace to False. This is required when the data doesn't fit in memory. If you don't use it your system will crash and you will need to start again. \n",
    "\n",
    "3. Increasing num_workers=cpus may also reduce the training time. \n",
    "\n",
    "\n",
    "⚠️ Note: bear in mind we are using dummy data. We are not intereted in accuracy. We are interested in time/epoch."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## np.memmap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I've tried different num_workers, but num_workers=0 and num_workers=cpus seem to have the same performance. \n",
    "\n",
    "⚠️ if you are running this notebook on Windows changing num_workers won't have any effect, as it will be disabled with this message: **Due to IPython and Windows limitation, python multiprocessing isn't available now.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>accuracy</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1.098613</td>\n",
       "      <td>1.098612</td>\n",
       "      <td>0.333885</td>\n",
       "      <td>1:06:40</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEXCAYAAABGeIg9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZwU1bXA8d+ZfYHZgRkYYNhXZURRxI24ghsaN4hJjNFHNDGILybB6NO45Zk8o++ZqIREojEKLrhFcd/QCAoom+yrDDAM0MwCzN7n/VE1Q8NsPVg9PdOc7+fTH6rurao+XTp9+t5bdUtUFWOMMSZQVLgDMMYY0/5YcjDGGNOAJQdjjDENWHIwxhjTgCUHY4wxDVhyMMYY04AlB2OMMQ1YcjCmESKyWUTODnccxoSLJQdjjDENWHIwJkgiEi8i/ysi293X/4pIvFuXJSKvi0ixiPhE5BMRiXLrfi0i20SkTETWiMhZbnmUiEwTkQ0iskdEnheRDLcuQUT+6ZYXi8hCEekWvk9vjjaWHIwJ3u3AaCAfGAGcCNzh1v0CKAC6AN2A3wAqIoOAm4BRqtoZOA/Y7O7zc+AS4AygO7AXeNStuwZIBXoCmcANQHnoPpoxh7LkYEzwrgbuUdUiVd0F3A38wK2rBnKA3qparaqfqDNxWS0QDwwVkVhV3ayqG9x9bgBuV9UCVa0EfgtcLiIx7vEygf6qWquqi1W1tM0+qTnqWXIwJnjdgS0B61vcMoD/AdYD74jIRhGZBqCq64GpOF/8RSIyW0Tq9ukNvOx2GxUDq3CSSTfgaeBtYLbbhfUHEYkN7ccz5iBLDsYEbzvOF3qdXm4Zqlqmqr9Q1b7AxcB/1o0tqOqzqnqqu68Cv3f33wqMV9W0gFeCqm5zWx93q+pQYAxwIfDDNvmUxmDJwZjmxLoDwwkikgDMAu4QkS4ikgXcCfwTQEQuFJH+IiJACU4LwC8ig0TkTHfgugJn3MDvHn86cL+I9HaP0UVEJrjL3xGRY0QkGijF6WbyY0wbseRgTNPm4nyZ170SgEXAMmA58CVwn7vtAOA9YB8wH3hMVT/EGW94ANgNFAJdgdvcff4PeA2nK6oMWACc5NZlAy/iJIZVwMc4XU3GtAmxh/0YY4w5nLUcjDHGNGDJwRhjTAOWHIwxxjRgycEYY0wDlhyMCQERyRMRde92NqbDseRgTDvk3ufwoYiUiMjmJrY5WUQ+c5fvFZHlIlIjIr9ty1hNZLLkYEz7tB+YCfyymW0uwLkXA5ypO34FvBHiuMxRwpKDOSqISHcRmSMiu0Rkk4hMCaj7rYi8KCLPudNqfykiIwLqh4jIR+4cSF+LyMUBdYki8kcR2eL+yv9URBID3vpqEflGRHaLyO3BxquqX6jq08DGZjY7Hzc5qOpTqvomUBbsexjTHEsOJuK5z1X4F7AU6AGcBUwVkfMCNpsAvABkAM8Cr4hIrDvZ3b+Ad3Dubv458Iw7FTfAg8DxOPMfZeD8eg+c5uJUYJD7nneKyBA3plPdyfaO9DPl4EzQ99WRHsOY5lhyMEeDUUAXVb1HVatUdSPwV2BiwDaLVfVFVa0GHsKZKmO0++oEPODu+wHwOjDJTTo/Bm52J8urVdXP3Om369ytquWquhQnOY0AUNVPVTXtW3ym84G31KY4MCFiV1KYo0FvoPthv9SjgU8C1rfWLaiqX0QKODgd91ZVDWwNbMFpgWThJJENNK0wYPkATqLxwvk4LRxjQsKSgzkabAU2qeqAZrbpWbfgtghycafjBnqKSFRAgugFrMWZTK8C6IfTKmgTblfXGcC1bfWe5uhj3UrmaPAFUOY+yzlRRKJFZLiIjArY5ngR+a57X8JUoBJnltTPcX7x/8odgxgLXATMdpPFTOAhd8A72r28NP7bBuw+XzoBiHVWJUFE4tzqU4FlgU+Gc2NLwPmbjnG3j/62cZijlyUHE/FUtRbnYTn5wCacX/x/w3lGc51XgatwnuP8A+C77gN3qnCSwXh3v8eAH6rqane/W3Gm714I+HAe5NPi35WInCYi+5rZ5HScacLn4rRUynEGxeHQS1jr/NXdZhLOs67LOfgIU2NazabsNkc996ax/qr6/XDHEgwRWQlcrqorwx2LiVzWcjCmA3G7lv5hicGEmg1IG9OBuN1cD4Q7DhP5rFvJGGNMA9atZIwxpoGI6FbKzMzUPn36hDuMoNTW1hId3TGuMLRYQ8NiDY2OEmt7inPx4sW7VbVLY3URkRx69uzJokWLwh1GUHw+HxkZGeEOIygWa2hYrKHRUWJtT3GKyJam6qxbyRhjTAOWHIwxxjRgycEYY0wDETHmYIwxrVVdXU1BQQEVFRVt+r5+v5+dO3e26XsmJCSQm5tLbGxs0Pu0mBxEZCbOvDRFqjq8kfrBwN+BkcDtqvpgS/uKyHM4D0ABSAOKVTVfRPKAVcAat26Bqt4Q9KcxxpggFRQU0LlzZ/Ly8hCRNnvfmpoaYmLa7ne5qrJnzx4KCgpozVWdwXQrPQmMa6beB0zBeSJWUPuq6lWqmq+q+cAc4KWA6g11dZYYjDGhUlFRQWZmZpsmhnAQETIzM1vdQmoxOajqPJwE0FR9kaouBKpbu684/1WuBGYFFa0xxngo0hNDnSP5nOEeczgN2Kmq6wLK+ojIV0ApcIeqftLYjiIyGZgMkJaTx7adu0iMbR83ljSnvLwcn6/JfNmuWKyhYbGGRmtj9fv91NTUhDCi9ve+rTk/4U4Okzi01bAD6KWqe0TkeJyHvA8LfKhJHVWdAcwAiM8ZoEmdUklPjjt8s3anPd0A0xKLNTQs1tBobaw7d+5s077/OnVjDsXFxTz77LP89Kc/bdX+559/Ps8++yxpaa17BHlUVFSrzk/YLmV1n7j1XeC5ujJVrVTVPe7yYpxn8w4M5ng2faAxpiMpLi7msccea1DeUqti7ty5rU4MRyKcLYezgdWqWlBXICJdAJ+q1opIX2AAsDFcARpjTKhMmzaNDRs2kJ+fT2xsLAkJCaSnp7N69WrWrl3LJZdcwtatW6moqODmm29m8uTJAOTl5bFo0SL27dvH+PHjOfXUU/nss8/o0aMHr776KomJiZ7EF8ylrLOAsUCWiBQAd+E81xZVnS4i2cAiIAXwi8hUYKiqlja2r6o+4R56Ig0Hok8H7hGRasAP3KCqQXWS2dTjxpgjdfe/vmbl9ga919/K0O4p3HXRsCbrH3jgAVasWMGSJUv46KOPuOCCC1ixYkX95aYzZ84kIyOD8vJyRo0axWWXXUZmZuYhx1i3bh2zZs3ir3/9K1deeSVz5szh+9/35oGGLSYHVZ3UQn0hkNvafVX1R42UzcG5tNUYY44qJ5544iH3ITzyyCO8/PLLAGzdupV169Y1SA59+vQhPz8fgOOPP57Nmzd7Fk+4B6Q9Y+0GY8yRau4XfltJTk6uX/7oo4947733mD9/PklJSYwdO7bR+xTi4+Prl6OjoykvL/csHptbyRhjwqBz586UlZU1WldSUkJ6ejpJSUmsXr2aBQsWtHF0kdRysKaDMaYDyczM5JRTTmH48OEkJibSrVu3+rpx48Yxffp0hgwZwqBBgxg9enSbxxc5ycE6lowxHcyzzz7baHl8fDxvvvlmo3V14wpZWVmsWLGivvzWW2/1NDbrVjLGGNNA5CQHazgYY4xnIic5GGOM8UzEJAdrOBhjjHciJjkYY4zxTsQkB7uU1RhjvBMxycEYYyJZp06dANi+fTuXX355o9uMHTuWRYsWefJ+EZMc7D4HY8zRoHv37rz44oshf5+ISQ7GGNORTJs2jUcffbR+/be//S333XcfZ511FiNHjuSYY47h1VdfbbDf5s2bGT58OOA8/W7ixIkMGTKESy+91NO5lSLnDmlrOBhjjtSb06BwubfHzD4Gxj/QZPVVV13F1KlT+dnPfgbA888/z9tvv82UKVNISUlh9+7djB49mosvvrjJZ0A//vjjJCUlsWrVKpYtW8bIkSM9Cz9ikoMxxnQkxx13HEVFRWzfvp1du3aRnp5OdnY2t9xyC/PmzSMqKopt27axc+dOsrOzGz3GvHnzmDJlCgDHHnssxx57rGfxRUxysIaDMeaINfMLP5SuuOIKXnzxRQoLC7nqqqt45pln2LVrF4sXLyY2Npa8vLxGp+puCzbmYIwxYXLVVVcxe/ZsXnzxRa644gpKSkro2rUrsbGxfPjhh2zZsqXZ/U8//fT6yftWrFjBsmXLPIstcloONuhgjOlghg0bRllZGT169CAnJ4err76aiy66iGOOOYYTTjiBwYMHN7v/jTfeyLXXXsuQIUMYMmQIxx9/vGexBfMM6ZnAhUCRqg5vpH4w8HdgJHC7qj7Y0r4i8lvgP4BdbtFvVHWuW3cbcB1QC0xR1beD+SCWG4wxHdHy5QcHwrOyspg/f36j2+3btw+AvLy8+qm6ExMTmT17dkjiCqZb6UlgXDP1PmAK8GAjdc3t+7Cq5ruvusQwFJgIDHP3e0xEooOI0RhjjIdaTA6qOg8nATRVX6SqC4Hq1u7biAnAbFWtVNVNwHrgxFbsb4wxxgPhHHO4SUR+CCwCfqGqe4EeQODDUgvcsgZEZDIwGSAuuz/FxcUkE55R/dYoLy/H52tNvgwfizU0LNbQaG2sfr+f6urqJu8hCBW/309NTU2bvqeq4vf7W3V+wpUcHgfuxbkC9V7gj8CPW3MAVZ0BzACIzxmgqalpZGQkeR2n53w+HxkZGeEOIygWa2hYrKHR2lhLSkooKSkhMzOzTRNETU0NMTFt99WrquzZs4ekpKRWnZ+wJAdV3Vm3LCJ/BV53V7cBPQM2zXXLjDHGU7m5uRQUFLBr166WN/aQ3+8nKqpt7yJISEggNze3VfuEJTmISI6q7nBXLwXqnpL9GvCsiDwEdAcGAF8Ec0ybeM8Y0xqxsbH06dOnzd+3o7TGgrmUdRYwFsgSkQLgLiAWQFWni0g2zrhBCuAXkanAUFUtbWxfVX0C+IOI5ON0K20GfuIe72sReR5YCdQAP1PVWg8/rzHGmCC0mBxUdVIL9YU43T9B76uqP2jmePcD97cUV8P9WruHMcaYptj0GcYYYxqImORgDQdjjPFOxCQHY4wx3omY5GAT7xljjHciJjkYY4zxTsQkB2s3GGOMdyImORhjjPFOxCQHG3IwxhjvRExysI4lY4zxTgQlB2OMMV6JmORg3UrGGOOdiEkOxhhjvBMxycEaDsYY452ISQ7GGGO8EzHJwcYcjDHGOxGTHIwxxngnYpKDPSbUGGO802JyEJGZIlIkIiuaqB8sIvNFpFJEbg1mXxH5HxFZLSLLRORlEUlzy/NEpFxElriv6d/mwxljjDkywbQcngTGNVPvA6YAD7Zi33eB4ap6LLAWuC2gboOq5ruvG4KID7AxB2OM8VKLyUFV5+EkgKbqi1R1IVAd7L6q+o6q1rirC2jiGdTGGGPCIybcAQA/Bp4LWO8jIl8BpcAdqvpJYzuJyGRgMkBcdn9KSkrwxdc0tmm7Ul5ejs/XZK5tVyzW0LBYQ6OjxNpR4gxrchCR24Ea4Bm3aAfQS1X3iMjxwCsiMkxVSw/fV1VnADMA4nMGaEpKKhkZKW0V+hHz+XxkZGSEO4ygWKyhYbGGRkeJtaPEGbarlUTkR8CFwNXqPuNTVStVdY+7vBjYAAwM5nh2tZIxxngnLMlBRMYBvwIuVtUDAeVdRCTaXe4LDAA2BnNMG5A2xhjvtNitJCKzgLFAlogUAHcBsQCqOl1EsoFFQArgF5GpwFBVLW1sX1V9AvgzEA+8KyIAC9wrk04H7hGRasAP3KCq7b9zzhhjIkyLyUFVJ7VQX0gTVxs1ta+q9m+ifA4wp6WYjDHGhFbE3CFtjDHGOxGTHGzMwRhjvBMxycEYY4x3IiY52KWsxhjjnYhJDsYYY7wTMcnBxhyMMcY7EZMcjDHGeCdikoM1HIwxxjsRkxyMMcZ4J2KSg9qggzHGeCZikoMxxhjvRExysHaDMcZ4J2KSgzHGGO9ETHKwIQdjjPFOxCQH61gyxhjvRFByMMYY45WISQ7WrWSMMd4JKjmIyEwRKRKRFU3UDxaR+SJSKSK3BrOviGSIyLsiss79N90tFxF5RETWi8gyERl5pB/OGGPMkQm25fAkMK6Zeh8wBXiwFftOA95X1QHA++46wHhggPuaDDweTIDWcDDGGO8ElRxUdR5OAmiqvkhVFwLVrdh3AvCUu/wUcElA+T/UsQBIE5GcYOI0xhjjjZgwvnc3Vd3hLhcC3dzlHsDWgO0K3LIdAWWIyGSclgVx2f0pKSnF12T6aj/Ky8vxdYRAsVhDxWINjY4Sa0eJM5zJoZ6qqoi0qmdIVWcAMwDicwZoSkoKGRkZIYnPSz6fr0PECRZrqFisodFRYu0ocYbzaqWddd1F7r9Fbvk2oGfAdrluWbNs4j1jjPFOOJPDa8A17vI1wKsB5T90r1oaDZQEdD8ZY4xpA0F1K4nILGAskCUiBcBdQCyAqk4XkWxgEZAC+EVkKjBUVUsb21dVnwAeAJ4XkeuALcCV7tvNBc4H1gMHgGuDidHaDcYY452gkoOqTmqhvhCn+yfofVV1D3BWI+UK/CyYuIwxxoSG3SFtjDGmgYhJDsYYY7wTMclBbdTBGGM8EzHJwXKDMcZ4J3KSgzHGGM9ETHKwhoMxxngnYpKDMcYY70RMcrBLWY0xxjsRkxyMMcZ4J2KSg13Kaowx3omY5GCMMcY7EZMcbMzBGGO8EzHJwRhjjHciJjlYw8EYY7wTMcnBGGOMdyImOdhjQo0xxjsRkxyMMcZ4p8XkICIzRaRIRFY0UT9YROaLSKWI3HpY3TgRWSMi60VkWkD5JyKyxH1tF5FX3PKxIlISUHdnsB/khUUFwW5qjDGmBcG0HJ4ExjVT7wOmAA8GFopINPAoMB4YCkwSkaEAqnqaquaraj4wH3gpYNdP6upU9Z5gP8gby3cEu6kxxpgWtJgcVHUeTgJoqr5IVRcC1YdVnQisV9WNqloFzAYmBG4gIinAmcArrQ3cGGNM6IRyzKEHsDVgvcAtC3QJ8L6qlgaUnSwiS0XkTREZFuybDeja6cgjNcYYc4iYML//JOBvAetfAr1VdZ+InI/TohjQ2I4iMhmYDJCc3ZfkWMHna7KB026Ul5d3iDjBYg0VizU0OkqsHSXOUCaHbUDPgPVctwwAEcnC6Xq6tK4ssAWhqnNF5DERyVLV3YcfXFVnADMAMnoPVomOJiMjw/tP4TGfz9ch4gSLNVQs1tDoKLF2lDhD2a20EBggIn1EJA6YCLwWUH858LqqVtQViEi2iIi7fKIb356W3kgAv9/uczDGGK+02HIQkVnAWCBLRAqAu4BYAFWdLiLZwCIgBfCLyFRgqKqWishNwNtANDBTVb8OOPRE4IHD3u5y4EYRqQHKgYkazN1tArV2E5wxxnimxeSgqpNaqC/E6TJqrG4uMLeJurGNlP0Z+HNLMR1OgJpaSw7GGOOVCLlDWvBby8EYYzwTEclBgFobczDGGM9ERHJALDkYY4yXIiI5CDYgbYwxXoqI5ICA3x/uIIwxJnJERHKwMQdjjPFWRCQHgBpLDsYY45mISA5il7IaY4ynIiI52NVKxhjjrYhIDjbmYIwx3rLkYIwxpoGISA428Z4xxngrIpKDTdltjDHeiojkAHYpqzHGeCkikoP7fCBrPRhjjEciIjnUsXEHY4zxRkQkB3H/tSuWjDHGGxGRHOpYcjDGGG8ElRxEZKaIFInIiibqB4vIfBGpFJFbD6sbJyJrRGS9iEwLKH9SRDaJyBL3le+Wi4g84m6/TERGthyf8691KxljjDeCbTk8CYxrpt4HTAEeDCwUkWjgUWA8MBSYJCJDAzb5parmu68lbtl4YID7mgw8HmSMNiBtjDEeCSo5qOo8nATQVH2Rqi4Eqg+rOhFYr6obVbUKmA1MaOHtJgD/UMcCIE1EcprbQdxRB7uc1RhjvBET4uP3ALYGrBcAJwWs3y8idwLvA9NUtbKJfXoAOwIPLCKTcVoWZHbvTSdgj28vUVVxnn8IL5WXl+PzNZln2xWLNTQs1tDoKLF2lDhDnRyacxtQCMQBM4BfA/cEu7OqznD3I2/QcAVISU0lIzXR+0g95PP5yMjICHcYQbFYQ8NiDY2OEmtHiTPUVyttA3oGrOe6ZajqDrfrqBL4O04XVLP7tKSm1rqVjDHGC6FODguBASLSR0TigInAawB14wji3N58CVB3JdRrwA/dq5ZGAyWquqPhoQ+qu1rJHvhjjDHeCKpbSURmAWOBLBEpAO4CYgFUdbqIZAOLgBTALyJTgaGqWioiNwFvA9HATFX92j3sMyLSBecetiXADW75XOB8YD1wALi2xfgAxQakjTHGK0ElB1Wd1EJ9IU73T2N1c3G+8A8vP7OJ7RX4WTBxHc4uZTXGGG9ExB3SdRPvWcvBGGO8ERnJwf3Xps8wxhhvRERyqGMtB2OM8UZEJIf6uZX8/vAGYowxESIykoP7r93nYIwx3oiI5FDXdLAxB2OM8UZEJIf6loMlB2OM8UREJQdrORhjjDciIjlQPyBtycEYY7wQEcnBupWMMcZbEZUcrOVgjDHeiIjkQP30GR37PofCkgpqajv2ZzDGRIaISA5RbtPh5tlLOuzke0WlFYz+7/d5+L214Q7FGGMiJTlI/fIby5t99EO7dfe/VgLw0pdBPdfIGGNCKkKSw8HljvrAn7qktntfJdpBP4MxJnJERHIIlBAbHe4QWq2sorp+ubpWeWtFYRijMcaYCEoO79xyOgAV1bVhjiQ463aW8eqSbdT6ldtfdp6Q+uAVI+iWEs/rHbRrzBgTOVpMDiIyU0SKRGRFE/WDRWS+iFSKyK2H1Y0TkTUisl5EpgWUP+OWr3CPH+uWjxWREhFZ4r7uDPaDdIp3HmrXEZLD19tLOOfhedw8ewn9fjOX15ZuByAnNYGT+mTy+UYfVTV21ZIxJnyCaTk8CYxrpt4HTAEeDCwUkWjgUWA8MBSYJCJD3epngMHAMUAicH3Arp+oar77uieYDwGQFOd0J+2vbP/J4cZ/ftloed8uyZx/TA6791Uyf+Mez993eUEJH64u8vy4xpjI02JyUNV5OAmgqfoiVV0IVB9WdSKwXlU3qmoVMBuY4O4zV13AFzTx/OnWSEmIJTZa2L2vkk2793PKAx/wX6+sYNPu/d/20J7rk5V8yPq08YPZ8LvzyUlNZOygLgBcM/MLNu7a1+QxXvqygGPuepv9lTVBvefK7aVc9OdPufbJhR2idWWMCa+YEB67B7A1YL0AOClwA7c76QfAzQHFJ4vIUmA7cKuqft3YwUVkMjAZoEePHhQX76W6Vnnsow1s3FnMtuJynl6whfdXFfL6T0Z6+LG+nfLyciqrqhie04k/XjqI372zkXP7d6KkeG+Dbc9/5BMev3IoI3p0blD3+zdXUVZZw0m/e48Pfj6KmMBLthrxxMcb65ffX7aFk/ukBRWrz9fk74J2xWINDYvVex0lzlAmh2A8BsxT1U/c9S+B3qq6T0TOB14BBjS2o6rOAGYA5Ofna0ZGRn3dW6sOdslsL6kkJqkzKQmxofkEreTz+SivFTI6JzKoVzZPXZ/dYJs/TTqOn8/6iopqP9c+s4JuKfH86rzBnDGoC1md4vloTRE7y6oA2FdZy5Z9wqi8jAbHCbSy6GuO753O0q3FfL2riguOz6Cqxs+o+99jylkDuO7UPo3GGnhe2zOLNTQsVu91lDhDebXSNqBnwHquWwaAiNwFdAH+s65MVUtVdZ+7PBeIFZGsYN9QmvjxvGxrSWviDrnS8mpSE5tOVheN6M6C286qX99ZWskvXljKCfe9x5RZX/Gjvy8E4N4JwwC4Yvp8yqua7ioqrahmTWEpp/TPYkTPND5dv5u5y3cw8I43KSmv5t7XV/J5CMY4jDEdVyiTw0JggIj0EZE4YCLwGoCIXA+cB0xS1frLckQkW8T5iheRE934gv7WOrx10DMjEYDqdjZfUUl5NamJzTfaslMTWHTH2Vx/ah/uvnhYfXndlU3H9EjlByfnEe12J727ameTx/rbvI34Fc4Z0o2zh3RjWUEJP33m0EHxV5ZsP9KPY4yJQMFcyjoLmA8MEpECEblORG4QkRvc+mwRKcBpAdzhbpOiqjXATcDbwCrg+YDxg+lAN2D+YZesXg6scMccHgEmaituF667nNWJC57+sTPEsXtfZbCH+FZKyqv57mP/5uWvCqj1KyUHDh+jh3nr9+LbX0XnILq5sjrFc8eFQ7lmTB5vTz2d//7uMfV1M380CoDPpp0JwPwNuxs9Rk2tn79/tpnxw7M5JjeVa8b0PqR+1n+MZvzwbN5Ytp19QQ5uG2MiX4tjDqo6qYX6Qpq42sjtGprbSHmj76uqfwb+3FJMTenXtRPbissBZxrvrinxAOzZX3Wkh2xWda2f37+5mutO60NOaiJvLNvBl98U8+U3xdzy3FIAnps8mn5dO3HdkwuZctYApr60GqDV9zEMyu7MoOzORAlkpybSpbPz2bqlJHDesG7MW7sbVUUO61v77uOfUVZRwzlDuwGQFBfDL84ZyB/fdSb469I5np+c0Y83VxQyZ3EB14zJ+zanxBgTISLmDmmAP008jh+Mdn4Z+9X5IgT42ycbm9vtiD3y/jr+9ukmfvnCMgBioxsOelw1YwEn3PceSwtKuO6pRfXl15/WcAA4GFeN6sUZA7scUnbGwK5sKy7nzx+sZ1txOcUHnGT4wqKtLCtwxluG5KTUb//jU/vwg9G9ueOCIfTrkkx+zzR6ZiTyyPvr6rcpKa/mxa8KW53EXlu6nbxpb/DQuza7rDEdWUQlh9SkWG6/YAgA3dxWA8DufVUUlVV49j5rCsv4zoMf8acP1gPOHc8Av3xxWVD733HBEHJSEz2L5/SBzpj9H99dyykPfMA5D8/jla+21ceT1SmewdkHL4dNjo/h3kuGc/1pfetbGikJsezZX8VFf/oUgOkfb+B3727iwXfWoKpBTYVeU+tnyqyvAA5JNMaYjieikgM4E+89e/1JzLlxDHBwUHrx5ob3ERwJv1+5/eXl9TfXnTm4K3sPVJM37Y36beoGkpfeeQP1UAAAABXOSURBVG592TUn9yYlIYZogQn5PTyJpU5uehKj+x68NG5XWSVPfLqpfn3q2QMadDcdrm48Y/m2Eu57fSVvu5P/zZi3kT63zaXvb+ayakdps8e4efaSQ9ZPvP899oaoS88YE1oRlxwAxvTPIjc9CYB3pp4BwEaP7pR+ftFWFm1xEs39lw7noStHHFL/0k/H1A8kpybF8umvv8Oa+8Zx94ThLPvteSz85cn14wVeevLaEw9ZX76thL5Zydxy9kAuP77lG9CPzU3jXHdc4m+fbmr0fP3k6cWN3l29e18lN/5zcf204yN7OTfYFZVVcty97/LPBVta/XmMMeEVkckhUGJcNElx0fg8+AWrqkx7aTng9OFffVJv0pLi6n91/+KcgRzX89A7j3PTk4iPCf004gmx0fxp0nGHlInAzWcPCHoa8xk/PIFrT8mrXz+lbxpTzuxPfEwU/zcxn298B3itkUteX1uynTfdlsaPxuQxe/LJjAg4D3e8soLSioZXbhlj2q+ITw4AGclxFJV9+8tZ1xc5cx31zEjk+Z+Mri+fdGIv1t8/np+f1XL3TShdNKI7mx+4oL4FcCRXHt110TBO7e+MYTxy2WBuOWcgK+8Zx8UjutMzI5FfzVlGyYHq+ntH/vLxBu55fWX9/qpKXEwUr/7sFP5w2bH15b9/c3WrY6mormVzO5wby5ijQbinz2gTx+am8uWWIx9z2Oo7wIPvrOG9lc6NZs9cN7rBfQox0e0nz8744QmNXtYarL9dcwKlFdVI9QFEhLqLsG76Tn9+PWc51/9jIQs372VA106sKzp0csC8gEkFrxzVkytH9eT6pxbxwiLnMtmB3RrOE9WU215azstfbeO9/zyD/l07HdFnMcYcmfbzjRZCA7t1ZltxOVNmfdWqR3Bu9R3gwj99wml/+JBXl2xnf1Utg7M70yszKYTReuPbtGASYqPp2jmhQflVo3px8YjuLHQH9wMTw/r7x/PgFSP44cl5Dfb77+8eQ1Wtn3MfnsfLXxUEFcMf3lrNy185s62818zd38aY0DgqksPZQ5xulteWbueJTzdRVBrcZa3XPbWQFdsOXqEzsFsnpo0fHJIYO4oLj80BYETPNK49JY8bzujH5gcuICY6isuPz62fziNQl87xnNA7HYBbnlsaVIJ+7KMN9cufrNvlUfTGmGAdFd1Kw3ukMufGMVz2+Gfc98YqHnp3LSvvafz5RZU1tZRV1FBYUsHanc4v4x+M7s1lx+eS37Plaa4j3TlDu/Hn7x3Haf27kJoU/Ey3j31/JCfe/z4Af5m3katO6El6clyj2z4QMD4xslcaCzftZVdZZUiu8jLGNO6oaDkADOt+8A7hA+4Mpo39gr3gkU854b73uNC9GWzq2QO495LhlhhcIsKFx3ZvVWIA6No5gU9//R3A+fKf9NcFjW730Ltrmf6x02r4+Zn9+e/vHktUFHznwY+4cvp8dgbZ6jPGfDtHTXJIiI1m3f3j+cHo3ojAsDvfos9tc3nonTXMWVxQfxfw+oB+9KxO8dz0nf5hjDqy5KYn8cmvnASxurCMvGlvcPy979Zf5lpV46+/+Q6c/2aDsjvz2NUj2VdZwxebfZz0u/ftSXbGtIGjJjkAxEZHMSG/O6qw3209PPLBen7xwlIWb9nLuf8775Dtp39/ZLu6CikS9MxIOmQK8j37qzj+3nd56J01DL3zLdbsLOO8Yd34v4n59fNPnTm4Gxt/d379eMbg/3orLLEbczQ56r75Tgh4YtoNZ/SrX758+vz6VsOPxuQx+fS+h2xrvHP1Sb0Y3TeDc4d2Iy4miupa5ZEP1lPjzt80Ki+DCfk9Drl5MCpKeOeW0+vXT3ngA/70/jpqg5jzyRjTekfFgHRTfn5mfyaf3pfLHv+sfq6kV352io0vhFhMdBSzJ58MwDd7DvDwe2vrL1v98/eO4/zhOY3u169LJ5768YlcM/MLthWX88d31/LFZh9PX3dSo9sbY47cUZkcnv2Pk/hik4/k+BiS4+FfPz+Va2Z+wbKC4kNmLzWh1ysziYevyufhq/Kp9Wujl8IGOmNgF+6+eBibdu9n7c4yPlm3mw9W7yS/a/t4RrgxkeKoTA5j+mUxpt/BR1N3io+pn8XVhE9LiaFO3bQgVTV+zn7oY3785CL+fcuJze9kjGmVoMYcRGSmiBSJyIom6geLyHwRqRSRWw+rGycia0RkvYhMCyjvIyKfu+XPuc+ZRkTi3fX1bn3ekX88E8niYqL4sTtR4BPzt4U3GGMiTLAD0k8Cjd815vABU4AHAwtFJBp4FBgPDAUmichQt/r3wMOq2h/YC1znll8H7HXLH3a3M6ZR14zJ4+IR3Zm5YBuPfbQ+3OEYEzGCSg6qOg8nATRVX6SqC4HD52U+EVivqhtVtQqYDUwQZ+KfM4EX3e2eAi5xlye467j1Z0k4pzo17ZqIcPsFQ+jWOY4/vLWGue4zJYwx306oxxx6AFsD1guAk4BMoFhVawLKexy+j6rWiEiJu/3uwAOLyGRgMkCPHj3w+ZrMXe1KeXm5xeqxWODp7w1iyssb+ekzX5KVHMsT3xtOz/SGkwe2Bx3lvILFGgodJc4OOyCtqjOAGQD5+fmakdEx7knw+XxYrKHx3A2nkH/vO+zeX83Ul9fy5s2nBf2go7bUkc6rxeq9jhJnqG+C2wb0DFjPdcv2AGkiEnNY+SH7uPWp7vbGNCs1KZYN95/P09edyKbd+7nluSU21YYxRyjUyWEhMMC9MikOmAi8ps6Mdx8Cl7vbXQO86i6/5q7j1n+grXkIgzmqRUUJpw3owthBXXhzRSGD/+ste4a1MUcgqG4lEZkFjAWyRKQAuAunqxdVnS4i2cAiIAXwi8hUYKiqlorITcDbQDQwU1W/dg/7a2C2iNwHfAU84ZY/ATwtIutxBsEnfvuPaY42j35vJJc+9m/W7tzHHa+s4O2vC9lVVsnvLzuW3PRE0pLigr6vwpijUVDJQVUntVBfiNM11FjdXGBuI+Ubca5mOry8ArgimLiMaUpyfAzv3HIG+ytruOTRf/PJOud6hgmP/huAY3qkctGIHNYU7mPOlwXccEY/fnneIEsYxrg67IC0McFIjo/hpjP7c/PsJYeUL99WwvJtJfXr0z/eUP8ciZP6ZDAouzPfH9076GdeP/v5N/zm5eUADOrWmf+6cCinDshqYS9j2i9LDibiHdfTeURpZnIcz/7HaErKq+melsCX3xTTrXM8g7I78/qyHdz3xkoqqv18vsnH55t8/GP+Fsb0y+Tui4cxoJEksWJbCbe+sJReGUm8s/Lgc653lJTz/Sc+Z0hOCkNzUrjlnAHkprf/544bE8iSg4l43dMSGJWXzk1nDmBQwMSKgV/Y3x/dm++d2IuoKOFAVQ0LNu7hgTdX89mGPZzz8Dwyk+OorPGTlhRLbnoi5wzN5qnPNvON7wCrC8sAWHH3eXSKj6HkQDUj7nmHVTtKWbWjlDlfFjDnxpMZ2SudN5bvIFmq+U5GBkVlFaQlxhEXc9TNnG86AEsOJuLFREfxwg0tT6wY5Y43JMXFcObgbpw5uBsrtpUw58sClmwt5qtvitlXWUPB3nIWbHRuYrrl7IGsLizlrCHd6BTv/DmlJsXy6s9OobrWT1Wtn588vZjLHp9/2LsdnKbs3gnDGN03k/5dO2GTAZj2wpKDMc0Y3iOV4T1SAaiormX+xj30ykjijpdX0DUlnsmn9yUxruGNdiMCngny9tTTOeehj9lfVUu3lHhGdO/EO6sP3rrzX69+Xb980YjuXHlCLqPyMkiIjaaiupbiA9Vkp7bPu71N5LLkYEyQEmKj+c6grgDMmjw66P26pyXy1tTTWbK1mAuOyaG4eC//c2Vn1haVMSI3jR0l5Xy4uojnFhXwr6Xb+dfS7QHvGUVFtZ++WcnkZSXzweoikuOiOXdYNtFRwph+mfj2VyEijMh1ElmUCDf8czGl5dWkJ8eRkhBLelIspw/swvG906mu9ZOWFNdorH6/Ulnjb5DwVJWqWj8l5dV06RRvLZyjgETC/WX5+fm6ZMmSljdsBzrKrfNgsYZKc7Eu3VrM84u28o3vAJ+s203XzvFcMyaPhZt9fLRmFwA5qQkUllYQzJ9uSkIM0VFCaUXNIY9UvfS4Htx54VAS46L5y8cbWbWjlL5dkpm/cQ/LCko4NjeVMf0yGZgRy9e7qpgxb2P9vhnJcYzslU58bBS+fVXExUQxsFsnjuuVTkZyHHv2VVFUVkFuehJ5mUnkpifhO1BFtAiJcdHERovzHPfKGnbtqyQ7JYFOCTHERUc1m3QqqmsRge3FFQiQm55Y/4z37cXlRFfvp2tWJoWlFRSVVjIkJ6Vdjue0p/9XRWSxqp7QWJ21HIxpR0b0TKvvkiosqaBL5/j6ey+2FZfj9ys9M5Ioq6hmy54DfLi6iAHdOpPfM42lBcXc+/pKCvaW88vzBvGjMXkku+MgB6pq+OeCLUz/eCO+/VW8/NU23lu1k8zkODbvOQA4D1tKiIliTL9M9lfWMP3jjQ2e0T0qL51eGcl8sXkPu8oqqaj2A7Bg4x7++smmb/XZO8XH0CMtEUWpqPazq6yS5PhoUhJjSUmIZUPRPsoqa+q3j4uJoltKPMX7qw8pP9xp7iXFVTV+0pPi6JGeSFxMFFECqk6LsEvneNKTnPfZWVZBTFQUB6pqSIiNpqyihl1llWzes5+qGj8x0VHERDmJLjE2mqS4aGKjo4iPiSIpLprEuBiS4qLJ6hTPgaoa9uyrIj7WqUuIjaamYj/dqmJJTYwlJTGGWPd4IkJFdS2FJRVERwkx0cKefVVERwnRUUJZRTXJ8THERDnbR0cJ8TFR+JX6bQSIjnbKW0q2LbGWQxtrT78aWmKxhkZ7iHVNYRm3PLeElTtKOWNgF568dhQHqmqdBOFOVlhUVsEdc5YQExvLXRcNIyZKyOwUX38MVa3/8qmq8fPVN3vZUVJB/66dSE2MZfe+Sr7xHWBbcTnJcc6X4IGqGqprFRGIiRKyUxMoLKlgZ2kF+ypr2FFSQa1fSUuKI6tTHBXVfkrLqykpr6aqxk9OWgIn5GWQEBPFuqJ9vLtyJ/sqa6iu9VN84OATA244ox/f+Pbz9tc78auSk5JAl87x7CqrdI5V68evECVQXdvyd6AI9EhLJCkumupapcbvp7yqlgNVtZRX1wbVimvp+Imx0Ryo8m4uMBGIjXaSYEyU86+TRKKIjnLKPrvtrCZbDpYc2lh7+GIIlsUaGu0l1gNVNSzavJcx/TLru2cO115iDcbuPXvw1cQxIOCqr8AE1pSK6lr2HqjCt7+KvfurSUuKRRXSkmLZX1VD54RYMpPjmpzhV1WpdcdqDlTVUl5Vy/6qGorKKkmKi6ZLp3iqag8mk12+YqLjk9h7oIp9lTXU1Prr901JiKV7WgIK1NQqmZ3i8PuVGr/SKSGGA5W11Pj91LplVTV+okSo9TvJzh8QS0V1LdW1Wl9W/1Klttb5949X5lu3kjHmUElxMZw+sEu4w/BMlEiDO9qD6VZJiI0mJzWRnNTEI3pfEacLKCY6qr4bD2BITuPb+9Kl3STcPzZT1/5Ga4wxxoSdJQdjjDENWHIwxhjTgCUHY4wxDVhyMMYY04AlB2OMMQ1YcjDGGNOAJQdjjDENRMQd0iJSBqwJdxxBygJ2hzuIIFmsoWGxhkZHibU9xdlbVRu9EzJS7pBe09Qt4O2NiCyyWL1nsYaGxeq9jhKndSsZY4xpwJKDMcaYBiIlOcwIdwCtYLGGhsUaGhar9zpEnBExIG2MMcZbkdJyMMYY4yFLDsYYYxro8MlBRMaJyBoRWS8i09pBPD1F5EMRWSkiX4vIzW55hoi8KyLr3H/T3XIRkUfc+JeJyMg2jjdaRL4Skdfd9T4i8rkbz3MiEueWx7vr6936vDaOM01EXhSR1SKySkRObsfn9Bb3v/0KEZklIgnt5byKyEwRKRKRFQFlrT6PInKNu/06EbmmDWP9H/f/gWUi8rKIpAXU3ebGukZEzgsoD/l3RGOxBtT9QkRURLLc9bCe16Cpaod9AdHABqAvEAcsBYaGOaYcYKS73BlYCwwF/gBMc8unAb93l88H3gQEGA183sbx/ifwLPC6u/48MNFdng7c6C7/FJjuLk8EnmvjOJ8CrneX44C09nhOgR7AJiAx4Hz+qL2cV+B0YCSwIqCsVecRyAA2uv+mu8vpbRTruUCMu/z7gFiHun//8UAf93shuq2+IxqL1S3vCbwNbAGy2sN5DfozheuNPfoPcjLwdsD6bcBt4Y7rsBhfBc7BuYM7xy3LwblxD+AvwKSA7eu3a4PYcoH3gTOB193/WXcH/PHVn1/3f/CT3eUYdztpozhT3S9cOay8PZ7THsBW9w88xj2v57Wn8wrkHfaF26rzCEwC/hJQfsh2oYz1sLpLgWfc5UP+9uvOa1t+RzQWK/AiMALYzMHkEPbzGsyro3cr1f0h1ilwy9oFt4vgOOBzoJuq7nCrCoFu7nI4P8P/Ar8C/O56JlCsqjWNxFIfp1tf4m7fFvoAu4C/u11gfxORZNrhOVXVbcCDwDfADpzztJj2eV7rtPY8tpe/ux/j/AKHdhiriEwAtqnq0sOq2l2sjenoyaHdEpFOwBxgqqqWBtap87MgrNcQi8iFQJGqLg5nHEGKwWmyP66qxwH7cbo/6rWHcwrg9tdPwElo3YFkYFxYg2qF9nIeWyIitwM1wDPhjqUxIpIE/Aa4M9yxHKmOnhy24fTp1cl1y8JKRGJxEsMzqvqSW7xTRHLc+hygyC0P12c4BbhYRDYDs3G6lv4PSBORujm3AmOpj9OtTwX2tEGc4PyCKlDVz931F3GSRXs7pwBnA5tUdZeqVgMv4Zzr9nhe67T2PIb1705EfgRcCFztJjOaiSlcsfbD+YGw1P0bywW+FJHsdhhrozp6clgIDHCvBInDGdB7LZwBiYgATwCrVPWhgKrXgLqrD67BGYuoK/+hewXDaKAkoIkfMqp6m6rmqmoeznn7QFWvBj4ELm8izrr4L3e3b5NfmKpaCGwVkUFu0VnAStrZOXV9A4wWkST3/4W6WNvdeQ3Q2vP4NnCuiKS7LaVz3bKQE5FxOF2hF6vqgcM+w0T36q8+wADgC8L0HaGqy1W1q6rmuX9jBTgXqhTSDs9ro8I12OHVC2fkfy3OFQm3t4N4TsVpli8Dlriv83H6kd8H1gHvARnu9gI86sa/HDghDDGP5eDVSn1x/qjWAy8A8W55gru+3q3v28Yx5gOL3PP6Cs7VHO3ynAJ3A6uBFcDTOFfQtIvzCszCGQupxvnCuu5IziNOf/9693VtG8a6Hqdfvu5va3rA9re7sa4BxgeUh/w7orFYD6vfzMEB6bCe12BfNn2GMcaYBjp6t5IxxpgQsORgjDGmAUsOxhhjGrDkYIwxpgFLDsYYYxqw5GCMMaYBSw7GGGMa+H+1U61DaT7tFAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "not enough values to plot a chart\n",
      "Total time        : 1:06:40.556316\n"
     ]
    }
   ],
   "source": [
    "tfms = [None, TSClassification()]\n",
    "batch_tfms = TSStandardize(by_sample=True)\n",
    "dls = get_ts_dls(X_large, y_large, splits=splits, tfms=tfms, batch_tfms=batch_tfms, inplace=False, bs=[512, 1_024], num_workers=0)\n",
    "learn = ts_learner(dls, InceptionTimePlus, metrics=accuracy, cbs=ShowGraph())\n",
    "timer.start()\n",
    "learn.fit_one_cycle(1, 1e-2)\n",
    "timer.stop()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>accuracy</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1.098606</td>\n",
       "      <td>1.098651</td>\n",
       "      <td>0.332850</td>\n",
       "      <td>1:07:33</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEXCAYAAACjyo8UAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZxcZZ3v8c+vqnpPZ+nuJB2ykASQLCwBBEFBMqAssuqAwIDjMmNGdIiojBfEK6DOHcbhOuqVMUSNiEKQCTAggigKxiUsCVsCAUkgSweSTrqyp9PdVfW7f5zTSSfp7urlVPfpzvf9evUrVec5p+pXJ93f89RTp55j7o6IiAxeif4uQERECktBLyIyyCnoRUQGOQW9iMggp6AXERnkFPQiIoOcgl5EZJBT0MugZ2arzOwD/V2HSH9R0IuIDHIKejkomVmJmX3HzN4Of75jZiVhW42ZPWJmW8wsbWZ/NLNE2Pa/zGydmW03s9fN7MxwecLMrjezlWbWYGb3mVlV2FZqZj8Pl28xs+fMbHT/vXo52Cjo5WB1I3AyMAM4FjgJ+GrY9iWgDhgJjAa+AriZHQn8M3Ciu1cCZwOrwm2uAS4GTgcOATYDt4dtHweGAeOBauAzQGPhXprIvhT0crC6Evi6u9e7+0bgFuBjYVsLMAY41N1b3P2PHkwKlQVKgGlmVuTuq9x9ZbjNZ4Ab3b3O3ZuAm4FLzCwVPl41cLi7Z919ibtv67NXKgc9Bb0crA4BVre5vzpcBvAfwArgN2b2ppldD+DuK4BrCUK83szuNbPWbQ4FHgyHZrYAywkODKOBnwGPA/eGw0TfMrOiwr48kb0U9HKwepsgnFtNCJfh7tvd/UvuPhm4EPhi61i8u9/j7qeG2zrw7+H2a4Fz3X14m59Sd18Xviu4xd2nAe8Fzgf+vk9epQgKejl4FIUfipaaWSkwH/iqmY00sxrga8DPAczsfDM73MwM2ErQM8+Z2ZFmdkb4oe1ugnH2XPj4c4B/NbNDw8cYaWYXhbf/xsyONrMksI1gKCeHSB9R0MvB4lGCYG79KQUWAy8DS4HngW+G6x4BPAHsABYB/+XuTxKMz98KbALWA6OAG8Jtvgs8TDDcsx14GnhP2FYLLCAI+eXAHwiGc0T6hOnCIyIig5t69CIig5yCXkRkkFPQi4gMcgp6EZFBTkEvkoeZTTQzD7/lKjLgKOhFCiw8j/5JM9tqZqs6WOcUM/tLePsbZrbUzDJmdnNf1iqDk4JepPB2AvOAf+lknfMIzvWHYPqFLwO/KnBdcpBQ0MuAY2aHmNn9ZrbRzN4ys9lt2m42swVm9otwKuHnzezYNu1TzeypcE6aV8zswjZtZWb2f81sddj7/pOZlbV56ivNbI2ZbTKzG7tar7s/6+4/A97sZLUPEQa9u//U3R8Dtnf1OUQ6o6CXASWcF/6XwEvAWOBM4FozO7vNahcB/w1UAfcA/2NmReFEYr8EfkPwrdZrgLvD6YcBbgNOIJiPpoqgV912qoJTgSPD5/yamU0Nazo1nMisp69pDMHkZy/09DFEOqOgl4HmRGCku3/d3Zvd/U3gh8DlbdZZ4u4L3L0F+DbBdAcnhz9DgFvDbX8PPAJcER5APgV8PpyILOvufwmnHG51i7s3uvtLBAeaYwHc/U/uPrwXr+lDwK9dX1OXAtFZBDLQHAocsl8POgn8sc39ta033D1nZnXsnYJ4rbu37aWvJnhnUENwQFhJx9a3ub2L4KARhQ8RvPMQKQgFvQw0a4G33P2ITtYZ33oj7KmPI5yCGBhvZok2YT8B+CvBRGW7gcMIeut9IhxOOh34ZF89pxx8NHQjA82zwPbw2q1lZpY0s6PM7MQ265xgZh8Jz3u/FmgimE3yGYKe+JfDMfuZwAXAvWHwzwO+HX7YmwxPeSzpbcHh9WRLgaLgrpWaWXHYfCrwctsrToW1lRL8fabC9ZO9rUMOXgp6GVDcPUtw4Y4ZwFsEPfEfEVyTtdVDwGUE1239GPCR8OIfzQTBfm643X8Bf+/ur4XbXUcwZfFzQJrgoiJ5/0bM7DQz29HJKu8nmBr5UYJ3EI0EHwjDvqdVtvphuM4VBNe2bWTvZQ5Fuk3TFMugEn7B6HB3v6q/a+kKM3sVuMTdX+3vWmTwUo9epJ+Ewzd3KeSl0PRhrEg/CYeSbu3vOmTw09CNiMggp6EbEZFBLpZDN9XV1T5p0qT+LiOvbDZLMjkwznpTrYWhWgtDtXbfkiVLNrn7yPbaYhn048ePZ/Hixf1dRl7pdJqqqqr+LqNLVGthqNbCUK3dZ2arO2rT0I2IyCCnoBcRGeQU9CIig1wsx+hFRLqjpaWFuro6du/e3efPncvl2LBhQ589X2lpKePGjaOoqKjL2yjoRWTAq6uro7KykokTJ2JmffrcmUyGVKpvotTdaWhooK6uju6cmdiVCZvmmVm9mS3roH2KmS0ysyYzu26/ti+El2tbZmbzwxn5REQitXv3bqqrq/s85PuamVFdXd3tdy5dGaO/Ezink/Y0MJvgMmxtCxobLn+3ux9FcHGIyw/cXESk9wZ7yLfqyevMG/TuvpAgzDtqr3f354CWdppTQFk4L3g5ey/+0KnmTC7/SiIi0iUFG1hy93VmdhuwhnD+bXf/TUfrm9ksYBZAzSETSKc7PLbERmNj44CoE1RroajWwuhurblcjkwmU8CKOn/uTZs2MX/+fK6++upubXvBBRfws5/9jOHDu3fJ4Vwu1639U7CgN7MRwEXAJGAL8N9mdpW7/7y99d19LjAXYPKUozwO3zTLJy7fiOsK1VoYqrUwulvrhg0b+uwD0f1lMhl27NjBHXfcwTXXXHNAW2d1PfbYYz16zkQi0a39U8jz6D9AcG3Pje7eAjwAvLeAzyci0i+uv/56Vq5cyYwZMzjxxBM57bTTuPDCC5k2bRoAF198MSeccALTp09n7ty5e7abOHEimzZtYtWqVUydOpVPf/rTTJ8+nbPOOovGxsbI6ivkIXANcLKZlRMM3ZwJxH8CGxEZ0G755Su8+va2/Ct2w7RDhnLTBdM7bL/11ltZtmwZL774Ik899RTnnXcey5Yt23MK5Lx586iqqqKxsZETTzyRv/3bv6W6unqfx3jjjTeYP38+P/zhD/noRz/K/fffz1VXRXOhtLxBb2bzgZlAjZnVATcRXOQYd59jZrUEAT4UyJnZtcA0d3/GzBYAzwMZ4AXCoZl8NEO+iAxkJ5100j7nuX/ve9/jwQcfBGDt2rW88cYbBwT9pEmTmDFjBgAnnHACq1atiqyevEHv7lfkaV8PjOug7SaCA0O36FooItJTnfW8+0pFRcWe20899RRPPPEEixYtory8nJkzZ7Z7HnxJScme28lkMtKhG811IyLSS5WVlWzfvr3dtq1btzJixAjKy8t57bXXePrpp/u4Ok2BICLSa9XV1bzvfe/jqKOOoqysjNGjR+9pO+ecc5gzZw5Tp07lyCOP5OSTT+7z+mIZ9Bq5EZGB5p577ml3eUlJSYenUbaOw9fU1LBs2d5ZZq677rp21++peA7dKOlFRCITy6B3Jb2ISGRiGfQiIhIdBb2IyCAXy6DXefQiItGJZdCLiEh0Yhn06tCLyGA3ZMgQAN5++20uueSSdteZOXMmixf3foqwWAa9iMjB4pBDDmHBggUFfY5YBr3G6EVkoLn++uu5/fbb99y/+eab+eY3v8mZZ57J8ccfz9FHH81DDz10wHarVq3iqKOOAoILrlx++eVMnTqVD3/4w5HNdxPLb8Zq8EZEeuyx62H90mgfs/ZoOPfWTle57LLLuPbaa/nc5z4HwH333cfjjz/O7NmzGTp0KJs2beLkk0/mwgsv7PC6rz/4wQ8oLy9n+fLlvPzyyxx//PGRlB/ToBcRGViOO+446uvrefvtt9m4cSMjRoygtraWL3zhCyxcuJBEIsG6devYsGEDtbW17T7GwoULmT17NgDHHHMMxxxzTCS1xTLo1Z8XkR7L0/MupEsvvZQFCxawfv16LrvsMu6++242btzIkiVLKCoqYuLEie1OUVxoecfozWyemdWb2bIO2qeY2SIzazKz69osP9LMXmzzsy28KEleGqMXkYHosssu495772XBggVceumlbN26lVGjRlFUVMSTTz7J6tWrO93+/e9//57J0ZYtW8bLL78cSV1d6dHfCXwfuKuD9jQwG7i47UJ3fx2YAWBmSWAd8GBXisop6UVkAJo+fTrbt29n7NixjBkzhiuvvJILLriAo48+mne/+91MmTKl0+2vvvpqPvnJTzJ16lSmTp3KCSecEEldXbnC1EIzm9hJez1Qb2bndfIwZwIr3b3zw1kop5wXkQFq6dK9HwTX1NSwaNGidtfbsWMHEFwgvHWK4rKyMu69997Ia+qr0ysvB+Z3deWckl5EJDIF/zDWzIqBC4Eb8qw3C5gFUFE7mXQ6XejSeq2xsXFA1AmqtVBUa2F0t9ZcLkcmkylgRfF67lwu16390xdn3ZwLPO/uGzpbyd3nAnMBhk+Y4lVVVX1QWu+k02kGQp2gWgtFtRZGd2vdsGEDyWSyw/PTCymTyZBK9d0JjO5OIpHo1v7pi6GbK+jGsE1AQzci0nWlpaU0NDTgg/xEDnenoaGB0tLSbm2X9zBkZvOBmUCNmdUBNwFF4ZPOMbNaYDEwFMiFp1BOc/dtZlYBfBD4p+69mG69BhE5yI0bN466ujo2btzY58+dy+VIJPpuNpnS0lLGjRvXrW26ctbNFXna1wPtPqu77wSqu1WRiEg3FRUVMWnSpH557oEwJBbPSc36uwARkUEklkEvIiLRiWXQa4xeRCQ6sQx6ERGJTiyD3jVKLyISmVgGvYiIRCeeQa8OvYhIZGIZ9Mp5EZHoxDLoRUQkOrEMep1eKSISnVgGvYiIRCeWQa8OvYhIdGIZ9Ip6EZHoxDLoNUYvIhKdWAa9iIhEJ5ZBrw69iEh08ga9mc0zs3ozW9ZB+xQzW2RmTWZ23X5tw81sgZm9ZmbLzeyUrhY22C8JJiLSV7rSo78TOKeT9jQwG7itnbbvAr929ynAscDyrhaWU86LiEQib9C7+0KCMO+ovd7dnwNa2i43s2HA+4Efh+s1u/uWrhaWVdKLiEQi7zVje2ESsBH4iZkdCywBPh9eR/YAZjYLmAVQXHs4mxoaKC1KFrC83mtsbCSd7vAYGCuqtTBUa2Go1mgVMuhTwPHANe7+jJl9F7ge+N/trezuc4G5ACVjjvBhw0dQUVLI8npvIFwUuJVqLQzVWhiqNVqFPOumDqhz92fC+wsIgr9LcvowVkQkEgULendfD6w1syPDRWcCr3Z1+1yuIGWJiBx08o6NmNl8YCZQY2Z1wE1AEYC7zzGzWmAxMBTImdm1wDR33wZcA9xtZsXAm8Anu1pYVj16EZFI5A16d78iT/t6YFwHbS8C7+5JYTrrRkQkGrH8ZixojF5EJCqxDXr16EVEoqGgFxEZ5GIb9Bq6ERGJRmyDXj16EZFoxDbo1aMXEYlGbIM+qy9MiYhEIsZBrx69iEgUYhv0GroREYlGbINePXoRkWjEN+jVoxcRiURsgz6nHr2ISCRiG/QauhERiUZ8g15DNyIikYht0OvCIyIi0cgb9GY2z8zqzWxZB+1TzGyRmTWZ2XX7ta0ys6Vm9qKZLe5OYerRi4hEoys9+juBczppTwOzgds6aP8bd5/h7t26AIk+jBURiUbeoHf3hQRh3lF7vbs/B7REWZi+MCUiEo28lxLsJQd+Y2YO3OHuczta0cxmAbMAimsPZ+u27aTTRQUur3caGxtJpzs8BsaKai0M1VoYqjVahQ76U919nZmNAn5rZq+F7xAOEB4E5gKUjDnCyysqqKqqKnB5vZNOp2NfYyvVWhiqtTBUa7QKetaNu68L/60HHgRO6uq2mr1SRCQaBQt6M6sws8rW28BZQLtn7rRHZ92IiEQj79CNmc0HZgI1ZlYH3AQUAbj7HDOrBRYDQ4GcmV0LTANqgAfNrPV57nH3X3e1MJ11IyISjbxB7+5X5GlfD4xrp2kbcGwP69IUCCIiEYntN2M1dCMiEo3YBr2GbkREohHboFePXkQkGrENevXoRUSiEdug14exIiLRiG/QK+dFRCIR26DX0I2ISDRiG/T6MFZEJBrxDXr16EVEIhHboNfQjYhINGIb9Bq6ERGJRmyDXj16EZFoxDbo1aMXEYlGLIPe0IVHRESiEsugB10cXEQkKnmD3szmmVm9mbV7dSgzm2Jmi8ysycyua6c9aWYvmNkjXS3KTGP0IiJR6UqP/k7gnE7a08Bs4LYO2j8PLO9eWaYxehGRiOQNendfSBDmHbXXu/tzQMv+bWY2DjgP+FF3ijLUoxcRiUreSwn20neALwOV+VY0s1nALICS2sPY1bibdLrD40ssNDY2xr7GVqq1MFRrYajWaBUs6M3sfKDe3ZeY2cx867v7XGAuQMXYd3mquISqqqpClReJdDod+xpbqdbCUK2FoVqjVcizbt4HXGhmq4B7gTPM7Odd3VhDNyIi0ShY0Lv7De4+zt0nApcDv3f3q7qyraEvTImIRCXv0I2ZzQdmAjVmVgfcBBQBuPscM6sFFgNDgZyZXQtMc/dtPa5Kp1eKiEQmb9C7+xV52tcD4/Ks8xTwVHcKU49eRCQasfxmrGGaj15EJCKxDHrQFAgiIlGJZdCb6QpTIiJRiWXQg2avFBGJSiyD3tDQjYhIVGIZ9KChGxGRqMQy6M3UoxcRiUosgx6dXikiEplYBn1wKUEFvYhIFGIZ9GjoRkQkMrEMevXoRUSiE8ugB8gq50VEIhHLoNelBEVEohPLoNcYvYhIdGIZ9BqjFxGJTt6gN7N5ZlZvZss6aJ9iZovMrMnMrmuzvNTMnjWzl8zsFTO7patFGUZGQS8iEomu9OjvBM7ppD0NzAZu2295E3CGux8LzADOMbOTu1KUGbRoVjMRkUjkDXp3X0gQ5h2117v7c0DLfsvd3XeEd4vCny51082gJaOgFxGJQkHH6M0saWYvAvXAb939mS5th9GsHr2ISCTyXjO2N9w9C8wws+HAg2Z2lLt3NNY/C5gFMGzMRJpasqTTHb6RiIXGxsbY19hKtRaGai0M1RqtggZ9K3ffYmZPEoz1txv07j4XmAtwyGHTPJODqqqqviivx9LpdOxrbKVaC0O1FoZqjVbBhm7MbGTYk8fMyoAPAq91bVs0dCMiEpG8PXozmw/MBGrMrA64ieCDVdx9jpnVAouBoUDOzK4FpgFjgJ+aWZLggHKfuz/SlaLMgmmKszknmbAevCwREWmVN+jd/Yo87euBce00vQwc15OiWqO9JZsjmUj25CFERCQUz2/Ghkmv4RsRkd6LZ9CHfXqdSy8i0nvxDHr16EVEIhProG/JaL4bEZHeimfQh/+qRy8i0nvxDPqwS9+sMXoRkV6LZ9CH/2oGSxGR3otn0LeO0SvoRUR6LaZBHyR9k4ZuRER6LZZB31pUY3O2X+sQERkM4hn0YVWNLQp6EZHeimXQt34zVj16EZHei2XQt05YqR69iEjvxTTowx69gl5EpNdiGfStp1fu0tCNiEivxTLoAcqKkuxWj15EpNfyBr2ZzTOzejPr6KLeU8xskZk1mdl1bZaPN7MnzexVM3vFzD7fncLKipPsas50ZxMREWlHV3r0dxJc1LsjaWA2cNt+yzPAl9x9GnAy8Dkzm9bVwsqKkjQ26wtTIiK9lTfo3X0hQZh31F7v7s8BLfstf8fdnw9vbweWA2O7WlhZcZLGFvXoRUR6K+81Y6NgZhMJrh/7TCfrzAJmAYwdO5YJKdi0rZF0usNjTL9rbIx3fW2p1sJQrYWhWqNV8KA3syHA/cC17r6to/XcfS4wF2DGjBk+algFdZt3UVVVVegSeyydTse6vrZUa2Go1sJQrdEq6Fk3ZlZEEPJ3u/sD3dm2uqKYzbuaC1OYiMhBpGBBb8EUlD8Glrv7t7u7fdWQYtI7m3HX5QRFRHoj79CNmc0HZgI1ZlYH3AQUAbj7HDOrBRYDQ4GcmV0LTAOOAT4GLDWzF8OH+4q7P9qVwqorimnJOtubMgwtLermyxIRkVZ5g97dr8jTvh4Y107Tn9h7sahuG1lZAsCGrbsV9CIivRDbb8ZOqCoHYO3mXf1ciYjIwBb7oF/doKAXEemN2AZ9VUUxFcVJBb2ISC/FNujNjEOrK1hRv6O/SxERGdBiG/QAJ02q4uk3G8hkNeeNiEhPxTro3zW6kkzO2bC9qb9LEREZsGId9ONGlAFQl9Y4vYhIT8U66MeGQb9uS2M/VyIiMnDFO+iHB0G/Nq2gFxHpqVgHfWlRktqhpazR0I2ISI/FOugBxleV6duxIiK9MACCvpy16tGLiPRY7IN+QlU567ftZndLtr9LEREZkGIf9IdWl+MOdZv1gayISE/EPugnVFUAsLphZz9XIiIyMOUNejObZ2b1Zrasg/YpZrbIzJrM7LrubNsVE6uDWSxXaXIzEZEe6UqP/k7gnE7a08Bs4LYebJtXVUUxQ0pSrFGPXkSkR/IGvbsvJAjzjtrr3f05oKW723aFmTGppoKVGxX0IiI9kfdSgn3FzGYBswDGjh1LOr33+DCpqpg/rtxMQ0MDwTXH46GxsXGfOuNMtRaGai0M1Rqt2AS9u88F5gLMmDHDq6qq9rQdN3EkDy/dSLaoglFDS/urxAOk02na1hlnqrUwVGthqNZoxf6sG4CpY4YC8Oo72/q5EhGRgUdBLyIyyHXl9Mr5wCLgSDOrM7N/MLPPmNlnwvZaM6sDvgh8NVxnaEfb9qTIYWVFAHzr16+zon57Tx5CROSglXeM3t2vyNO+HhjXk227Y9qYobz6zjb+7dHX+PEnTozqYUVEBr0BMXQD8KvZpzKltpLn12wml/P+LkdEZMAYMEFvZvzjaZPZvKuF5es1Vi8i0lUDJugBZh45klTCeOjFt/u7FBGRAWNABX3NkBLOnDqKnz+9mqbM4J62+OW6Lcye/wKfu+d53tigD6BFpOcGVNADXHbieHY1Z/nc3S/gPjjH6ne3ZPnSfS/x8Etv86uX3+GD/7mQh15c199licgANeCC/m+OHMUHpo7mieUb+OJ9L/V3OZFzd258cBlv1O/g7OmjufmCaQB8/t4X+elfVvVvcSIyIA24oDczvv93xwHw4Avr2NWc6eeKorOzKcOVP3qG+5+vA+CzMw/nE++bxDNfOZOTJlVx08OvcNeiVf1ao4gMPAMu6AFKi5L8YtbJAMx/dm0/V9Mz7s4Tr27gO0/8lXufXcPPn17NBd//E39Z2QDAPZ9+D8eOHw7A6KGl/PSTJ3HSxCq+9tAr/Odv/zpoh61EJHqxmdSsu94zuZqTJlZx+5MrOGVyNdMOGdrfJXXLopUN/ONdi9ttu+tTJ/Hew2r2WVZWnOSeT7+HGx5Yynd/9wYbdzTxjYuOIpmIz2yeIhJPA7JH3+r/fORoSlIJPn3XYtI7m/u7nG752LxnAfj4KYdy2hE1HDch6L3f8bETeP+7Rra7TSqZ4FuXHMNnZx7GPc+s4bN3L6FhR1Of1SwiA9OA7dEDHD5qCD+46gQuvv3PHP+N3/K186fxgamjmRBefrAzz76VJpPNccph1d2a437LrmZWNexi7PAykj2s+8d/eotszplQVc4tFx3VrW3NjC+fM4WaISV8/ZFXeWJ5PTPfNZLPzDyMEyfGe6pUEekfAzroAWaMH84/nT6ZO/7wJl9/5FW+/sirzD7jcC4+biyTair2CXF3Z3XDLq6Z/wJL120FYEJVOe89rJrLThxP7bBSxgwra/d5Nu9s5iM/+Atvbdp7pauPnTiG686tZFh5UZfr/c4Tf+U7T7wBwJyrTujJSwbgU6dO4tQjarj/+TruX7KOS+csYvLICr52/jTqtwW9/OHlRXxg6mgS+w3vNGWyJMwoSg7oN3Qi0kUWxw/1ZsyY4S+++GK3tmnKZLnnmTXc8stX9yy76uQJfPW8aZQWBX3vXzy3hv91/1IAzpgyionVFTz80tts2m/44wNTR3PaETW87/Aatja2MOuuxTS0GRqaUlvJkJIUS1ZvxoHaoaV8+7JjOXzUEEZVdnxhFHfn+G/8lpGVJXz7ozM4auywbr3GjjQ2Z/npolXc+thrB7QNLU1x/blTOX1iGTXVVdz551X822OvUZJKcPOF0zlz6qhOa+4PA+FCDq1Ua2Go1u4zsyXu/u522wZL0Ldq2NHET/68imfeauC5VZtJJoykGRg0Z3IA3HLhdD7+3olAEL6Pv7KBrzy4FCM4o2fdlsZ2H/ufTp/MDedO3XN/0fK13PirlbzZppd/2hE1XDxjLO9/10jKi5MUJRMUpxJs293CL196mxsfXMZ/XHIMl757fI9eX2dWN+zkxgeXcc5RtQwpSfHQi+t48vWNebcbUV7Egqvfy2Ejh0ReU0/E5Q+nK1RrYajW7juogr6Vu/OnFZv43fJ6tu/OsHh1mnEjyvjSWUdy/IQRnW6bzTmPLn2HOX9YyYr6HZw5dRT/74rjDzjDpfU/+Nm30tzyy1eYWF3Br19ZTzbP7Jpv/Ou5fTZs0tic5dGl7/Cn19/hlQ27+OuGHfzhX2YydngZDzy/jseWvbPPwaCyNMXkmgqunnkYZ0+vzfv5hbuzdN1Wxgwro7qiGIdenwkUlz+crlCthaFau69XQW9m84DzgXp3P+CTQzObAvwEOB640d1va9N2DvBdIAn8yN1v7UrBUQR9X2jvP3hHU4aV9TtY+NeNe66I9XLdVtZtaaSyJMVd/3ASx+U50PRVra3Wb93NVT9+hhX1O5hSW0nd5kZ2NGU44dARTB1TybgR5Zx6eA3TDxm6J/h3t2Sp29zINfNfYHmbK3/VDCnmg9Nq+buTJjD9kKGsTu9iaGmK6iElkdQaN6q1MFRr93UW9F35MPZO4PvAXR20p4HZwMX7PWkSuB34IFAHPGdmD7v7qwc+xOAxpCTFseOH7/my00BQO6yUR2efxjtbGzm0uoJszvnpX1bxvd+/wZLVm/esN7KyhNOOqGHVpp08v2bLPo9x+rtGYgYlqQQPPF/H/GfX7NN+yuRqzpo+mvdMqmZKbeUBHxCLSOF05QpTC81sYtvE8nMAAAs+SURBVCft9UC9mZ23X9NJwAp3fxPAzO4FLgIGddAPVMWpBIdWVwDB0MunTp3Ex987kUwux+vrt/Pq29u4a9FqHng+mFytsiTF9qYM5x0zhtsuOZay4r0nm27Z1cwTy+v59bL1lBUnWd2wk1UNO/d8UD5mWCmnv2skZ0+v5cRJVQwpGfAnf4nEWiH/wsYCbecnqAPe09HKZjYLmAUwduxY0ul0AUuLRmNj44CoE3pX67hyGHf4EM46fDpr0o1UVxRTnDLeamhk7LBSGndsZf+Pr8+YVM4Zkybvs2z9tiaeXb2V3/01zb3PreXe59ZiwGEjy5k6uoJptUMYVprCci1MHr2TccNK9zmAxNHB8jvQ11RrtGLTlXL3ucBcCMbo4zDmlU9cxua6Iqpa2z7E6Pa/wNvpttMmjuETpwdnRz39ZprX1m9j6bqt/Pmtrfxy2YFnCI2qLGFCVTkTqsupKi9mSGmKISUp1m1pZEX9DsyMytIU40eUM25EGaMqSxhWVkTWnWzOyWQdDLJZp7QoycjKEopTCUZWllCaSpAKPxRvyebI5py3Nu2ktCjJsLIiKktTFCUTNGWypBIJEsYBH04X4neg9XOz7nyRrysOxt/XvjAQai1k0K8D2p5DOC5cJkL1kBLOO2YM5x0zBgjC7a1NO9nVnGXzlq1szqRY07CTVQ27WJPexaKVDWxtbGFXc3DBmeJkgmHlRZSkEqQSxm9eWU9LtvtnkA0tTbE7k9tz6u3+kgkjm3MSFpx6O3poKSPKixhVWUpFSYqmpiZSRUW8s3U35cVJkokEZsFQ2NDwoFSUTJAwI2FA+G/r/ZJUksNGVfDkaxt5YvkGmjI50jubKUklKC9Oksk5ZUVJSouSpJKGOxQljcrS4EA0pCTF8PIiaoaUsLMpQyYXHNDKi5KUl6QoSSUoTiZIJY2mxl2MGNZEKhHcL04mGFFRzM6mDLZPXUbbY0zrbWPv8naXtVnOAcvtgHXabptzJ+fB74EDW7Y00tCynaz7nn1lYW0J27ttKmkkE0YqkSCTzdGczVGUDP4PCH8dWn8rfM993+f+/qzN41tYdMKMXNh5yOYc9+B3oziVYGdThvKWLMWtz7vncaI5UEdx4C9k0D8HHGFmkwgC/nLg7wr4fDKAmRmTw/P402XZDntI2ZyzoylDWVGS4lRin+Ubtu2mYUczq9M7WVG/gyNGVVIzpJhUMvgW8K7mLOs2B4NMm3Y0sbM5y9ZdzZQWBd93yLozdcxQMtkc2xpb2NqYYWtjC8Wp4A+4OZNj/bbdbNnVzBv129ndkiOXy5FMJhhVWcLGHRlyuSC0mjM5tjdl2L67hUw2iJace4fhkjA4e3otVRXFDCsrorElSybrJBPG9t0ZmjLZ4ICTMDLZHNt3Z9i8s5k1DbvYvKuZzbtaKEoaJakkzZkg8KT/7T1A0e7B1IBEYu8BrCXre/7/cn7gASmVsD2/A20fJ5HnIJA36M1sPjATqDGzOuAmoCgowOeYWS2wGBgK5MzsWmCau28zs38GHic4vXKeu7/Szf0kso9kwhhWduCUE8mEccjwMg4ZXsbR46L5xnFXdPdtu4dh3xr8DTuaWbcl+NxjYk1Fj+vY2ZTZ8+U8CA5Ku5ozNGdytOSclkyOTZs3U14xlEwuR0vWacpk2bi9iaFlRRiEdTmtx4jW3jW0hs3e4Gmvl+xtetBtT9tud519HiM4oLXtSe/csZPKyiEkE8G7mKDHH2zf+j2V1h52S87JZnOkksG7l5Zcbs/ztH33sO/9fdvb1upt9sXe+8E7i9YvYJoFdTRnc2zZtoNkcemed4Vm7LNda92t71ha3720Lvc291Phu4TiVIKkGc7ed0ROMMSYc6cokcDZu10u53y1k9+Prpx1c0We9vUEwzLttT0KPJrvOUQOFtZmWCSJUTuslNphvZ+ComK/M5eCsCjeZ9nQRBNVVQNjOu+BMO7dKi61dhb0mtVKRGSQU9CLiAxyCnoRkUFOQS8iMsgp6EVEBjkFvYjIIKegFxEZ5BT0IiKDXCyvMGVm24HX+7uOLqgBNvV3EV2kWgtDtRaGau2+Q9293akGYzN75X5e7+hKKXFiZosHQp2gWgtFtRaGao2Whm5ERAY5Bb2IyCAX16Cf298FdNFAqRNUa6Go1sJQrRGK5YexIiISnbj26EVEJCIKehGRQS5WQW9m55jZ62a2wsyuj0E9483sSTN71cxeMbPPh8urzOy3ZvZG+O+IcLmZ2ffC+l82s+P7uN6kmb1gZo+E9yeZ2TNhPb8ws+JweUl4f0XYPrGP6xxuZgvM7DUzW25mp8R4n34h/L9fZmbzzaw0LvvVzOaZWb2ZLWuzrNv70cw+Hq7/hpl9vA9r/Y/wd+BlM3vQzIa3abshrPV1Mzu7zfKCZ0R7tbZp+5KZuZnVhPf7db92WXBps/7/Ibjc4EpgMlAMvERwScL+rGkMcHx4uxL4KzAN+BZwfbj8euDfw9sfAh4juPrXycAzfVzvF4F7gEfC+/cBl4e35wBXh7c/C8wJb18O/KKP6/wp8I/h7WJgeBz3KTAWeAsoa7M/PxGX/Qq8HzgeWNZmWbf2I1AFvBn+OyK8PaKPaj0LSIW3/71NrdPCv/8SYFKYC8m+yoj2ag2Xjye4NOpqoCYO+7XLr6m/nridnXsK8Hib+zcAN/R3XfvV+BDwQYJv7Y4Jl40h+IIXwB3AFW3W37NeH9Q2DvgdcAbwSPiLt6nNH9Ke/Rv+sp4S3k6F61kf1TksDE/bb3kc9+lYYG34x5oK9+vZcdqvwMT9wrNb+xG4ArijzfJ91itkrfu1fRi4O7y9z99+637ty4xor1ZgAXAssIq9Qd/v+7UrP3Eaumn9o2pVFy6LhfBt+HHAM8Bod38nbFoPjA5v9+dr+A7wZSC8tDPVwBZ3z7RTy546w/at4fp9YRKwEfhJOMz0IzOrIIb71N3XAbcBa4B3CPbTEuK5X1t1dz/G5e/uUwQ9Y4hhrWZ2EbDO3V/aryl2tbYnTkEfW2Y2BLgfuNbdt7Vt8+Bw3a/nqJrZ+UC9uy/pzzq6KEXwtvgH7n4csJNgiGGPOOxTgHB8+yKCg9MhQAVwTr8W1Q1x2Y/5mNmNQAa4u79raY+ZlQNfAb7W37X0VJyCfh3BGFirceGyfmVmRQQhf7e7PxAu3mBmY8L2MUB9uLy/XsP7gAvNbBVwL8HwzXeB4WbWOp9R21r21Bm2DwMa+qBOCHo2de7+THh/AUHwx22fAnwAeMvdN7p7C/AAwb6O435t1d392K9/d2b2CeB84MrwwEQnNfVXrYcRHOxfCv/GxgHPm1ltDGttV5yC/jngiPCMhmKCD7Me7s+CzMyAHwPL3f3bbZoeBlo/Rf84wdh96/K/Dz+JPxnY2uZtdMG4+w3uPs7dJxLst9+7+5XAk8AlHdTZWv8l4fp90vNz9/XAWjM7Mlx0JvAqMdunoTXAyWZWHv4utNYau/3aRnf34+PAWWY2InwHc1a4rODM7ByC4cYL3X3Xfq/h8vAspknAEcCz9FNGuPtSdx/l7hPDv7E6gpM01hPD/dqu/vpwoIMPQD5EcGbLSuDGGNRzKsFb35eBF8OfDxGMu/4OeAN4AqgK1zfg9rD+pcC7+6Hmmew962YywR/ICuC/gZJweWl4f0XYPrmPa5wBLA736/8QnJUQy30K3AK8BiwDfkZwJkgs9iswn+CzgxaC8PmHnuxHgvHxFeHPJ/uw1hUE49itf1tz2qx/Y1jr68C5bZYXPCPaq3W/9lXs/TC2X/drV380BYKIyCAXp6EbEREpAAW9iMggp6AXERnkFPQiIoOcgl5EZJBT0IuIDHIKehGRQe7/A7lgrwgrH6R0AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "not enough values to plot a chart\n",
      "Total time        : 1:07:33.940965\n"
     ]
    }
   ],
   "source": [
    "tfms = [None, TSClassification()]\n",
    "batch_tfms = TSStandardize(by_sample=True)\n",
    "dls = get_ts_dls(X_large, y_large, splits=splits, tfms=tfms, batch_tfms=batch_tfms, inplace=False, bs=[512, 1_024], num_workers=cpus)\n",
    "learn = ts_learner(dls, InceptionTimePlus, metrics=accuracy, cbs=ShowGraph())\n",
    "timer.start()\n",
    "learn.fit_one_cycle(1, 1e-2)\n",
    "timer.stop()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case, GPU utilization is very variable, although a lot of the time stays at 0%. That is because the dataloader takes time to build the batches because indexing is pretty slow with np.memmap arrays with such a large dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## zarr arrays"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We recommend to set num_workers = cpus. That will significantly speed up your training, as zarr array reading can be done in parallel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>accuracy</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1.098604</td>\n",
       "      <td>1.098629</td>\n",
       "      <td>0.332995</td>\n",
       "      <td>46:36</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEXCAYAAACkpJNEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU1d348c93JpNMVrJAwr6vKSACKtYNigu478hPrXUpjz5aXGotVluXah9prY/1qUuxpS5VUHFXXOpWXEBZBETWsEnYAgkJCUzIMuf3x70Jk5BlAjOZuTPf9+uVFzP33HvnOzfkfu8959xzxBiDUkqp+OOKdABKKaUiQxOAUkrFKU0ASikVpzQBKKVUnNIEoJRScUoTgFJKxSlNAEopFac0Aai4JSKbROTUSMehVKRoAlBKqTilCUCpACKSJCKPisg2++dREUmyyzqKyDsiUioiJSLyuYi47LJfi8hWESkXkTUiMt5e7hKRaSKyXkSKReRlEcm2y7wi8i97eamILBSRvMh9exVvNAEo1dBdwBhgBHAUcCxwt132S6AQ6ATkAb8BjIgMAm4CjjHGpANnAJvsbX4BnA+cAnQF9gCP22VXAR2AHkAOcD3gC99XU6ohTQBKNXQ5cL8xpsgYswu4D7jSLqsGugC9jDHVxpjPjTWYVi2QBOSLiMcYs8kYs97e5nrgLmNMoTHmAHAvcLGIJNj7ywH6G2NqjTGLjTF72+2bqrinCUCphroCmwPeb7aXAfwJKAA+FJENIjINwBhTANyCdXIvEpHZIlK3TS/gdbuKpxRYhZUw8oDngQ+A2XZ10x9FxBPer6fUQZoAlGpoG9ZJu05PexnGmHJjzC+NMX2Bc4Hb6ur6jTEvGmNOtLc1wHR7+y3ARGNMZsCP1xiz1b6LuM8Ykw/8GDgb+Gm7fEul0ASglMdujPWKiBeYBdwtIp1EpCPwO+BfACJytoj0FxEByrCu5P0iMkhEfmI3Fldi1eP77f0/BTwoIr3sfXQSkfPs1+NEZJiIuIG9WFVCfpRqJ5oAVLybi3XCrvvxAouA5cB3wBLgAXvdAcBHQAUwH3jCGPMpVv3/Q8BuYAeQC9xpb/MX4C2saqNyYAFwnF3WGZiDdfJfBfwHq1pIqXYhOiGMUkrFJ70DUEqpOKUJQCml4pQmAKWUilOaAJRSKk5pAlDqCIhIbxEx9pO9SjmKJgClIsh+FuBTESkTkU3NrHO8iHxlv/69iHwnIjUicm97xqpijyYApSJrHzAT+FUL65yF9bwCWENR3AG8G+a4VBzQBKBiioh0FZFXRWSXiGwUkakBZfeKyBwRecketnmJiBwVUD5ERD6zx+35XkTODShLFpE/i8hm+2r9CxFJDvjoy0XkBxHZLSJ3BRuvMeYbY8zzwIYWVjsTOwEYY541xrwHlAf7GUo1RxOAihn22PxvA8uAbsB44BYROSNgtfOAV4Bs4EXgDRHx2IOwvQ18iPUk7y+AF+yhngEeBkZhjdmTjXUVHjhsw4nAIPszfyciQ+yYTrQHgTvc79QFa+C4bw93H0o1RxOAiiXHAJ2MMfcbY6qMMRuAp4HLAtZZbIyZY4ypBh7BGvphjP2TBjxkb/sJ8A4w2U4s1wA324O41RpjvrKHd65znzHGZ4xZhpWAjgIwxnxhjMk8gu90JvC+0Uf2VRhozwUVS3oBXRtdcbuBzwPeb6l7YYzxi0ghB4d73mKMCbyq34x1J9ERK1Gsp3k7Al7vx0omoXAm1p2KUiGnCUDFki3ARmPMgBbW6VH3wr6y74493DPQQ0RcAUmgJ7AWa5C3SqAf1tV9u7CrpU4Brm6vz1TxRauAVCz5Bii35+dNFhG3iAwVkWMC1hklIhfa/fZvAQ5gjdD5NdaV+x12m8BY4Bxgtp0QZgKP2I3MbrtrZtKRBmzPGewFPNZb8YpIol18IrA8cJYwOzYv1t9ugr2++0jjUPFJE4CKGcaYWqxJVUYAG7Gu3P+ONe9unTeBSVhz814JXGhPzFKFdcKfaG/3BPBTY8xqe7vbsYaHXgiUYE340urfj4icJCIVLaxyMtYw1HOx7jh8WA3R0LD7Z52n7XUmY81f7OPglJVKtYkOB63ihv3gVH9jzBWRjiUYIrISuNgYszLSsajYpHcASkUhuxroOT35q3DSRmClopBdJfVQpONQsU2rgJRSKk5pFZBSSsUpR1UB5eTkmD59+kQ6jKDU1tbidjujd57GGnpOiRM01nCJplgXL1682xjTqfFyRyWAHj16sGjRokiHEZSSkhKys7MjHUZQNNbQc0qcoLGGSzTFKiKbm1quVUBKKRWnNAEopVSc0gSglFJxylFtAEop1VbV1dUUFhZSWVnZrp/r9/vZuXNnu36m1+ule/fueDyeoNbXBKCUimmFhYWkp6fTu3dvRKTdPrempoaEhPY7xRpjKC4uprCwkGB7S2oVkFIqplVWVpKTk9OuJ/9IEBFycnLadKejCUApFfNi/eRfp63f01EJYG9lTaRDUEqpmOGoBFCmCUAp5TClpaU88cQTbd7uzDPPpLS0tPUVj4CjEgA6bp1SymGaSwA1NS1f0M6dO5fMzMxwhQVoLyCllAqradOmsX79ekaMGIHH48Hr9ZKVlcXq1atZu3Yt559/Plu2bKGyspKbb76ZKVOmANC7d28WLVpERUUFEydO5MQTT+Srr76iW7duvPnmmyQnJx9xbJoAlFJx4763v2fltr2tr9gG+V0zuOecHzVb/tBDD7FixQqWLl3KZ599xllnncWKFSvqu2rOnDmT7OxsfD4fxxxzDBdddBE5OTkN9rFu3TpmzZrF008/zaWXXsqrr77KFVcc+cR2jkoAWgOklHK6Y489tkE//ccee4zXX38dgC1btrBu3bpDEkCfPn0YMWIEAKNGjWLTpk0hicVRCUAppY5ES1fq7SU1NbX+9WeffcZHH33E/PnzSUlJYezYsU32409KSqp/7Xa78fl8IYnFWY3ASinlMOnp6ZSXlzdZVlZWRlZWFikpKaxevZoFCxa0a2x6B6CUUmGUk5PDCSecwNChQ0lOTiYvL6++bMKECTz11FMMGTKEQYMGMWbMmHaNrdUEICIzgbOBImPM0CbKBwP/BEYCdxljHm5tWxH5E3AOUAWsB642xoS3w6tSSkXIiy++2OTypKQk3nvvvSbL6ur5O3bsyIoVK+qX33777SGLK5gqoGeACS2UlwBTgYebKGtu238DQ40xw4G1wJ1BxKGNwEopFUKtJgBjzDysk3xz5UXGmIVAdbDbGmM+NMbUPQWxAOgeVLSaAZRSKmSioQ3gGuCl5gpFZAowBSC1c19KSprNRVHF5/NprGHglFidEifEfqx+v7/Vp27DIZKfG+wximgCEJG7gBrghebWMcbMAGYAZPUcbKJlkuXWRNOE0K3RWEPPKXFC7Me6c+fOdh2Xv057zwdQx+VyBX2MIpYARORnWA3E440xQVXuaA2QUkqFTkQSgIhMAO4ATjHG7I9EDEopFe9abQQWkVnAfGCQiBSKyLUicr2IXG+XdxaRQuA24G57nYzmtrV3+1cgHfi3iCwVkaeCC1fvAZRSsS0tLQ2Abdu2cfHFFze5ztixY1m0aNERf1ardwDGmMmtlO+gmV48zW1rjOkfVHRKKRWnunbtypw5c8L6GToUhFJKhdG0adN4/PHH69/fe++9PPDAA4wfP56RI0cybNgw3nzzzUO227RpE0OHWs/P+nw+LrvsMoYMGcIFF1wQsrGAoqEbaNCCaypWSqlmvDcNdnwX2n12HgYTH2q2eNKkSdxyyy3ceOONALz88st88MEHTJ06lYyMDHbv3s2YMWM499xzm53T98knnyQlJYVVq1axfPlyRo4cGZLQHZUAlFLKaY4++miKiorYtm0bu3btIisri86dO3Prrbcyb948XC4XW7duZefOnXTu3LnJfcybN4+pU6cCMHz4cIYPHx6S2DQBKKXiRwtX6uF0ySWXMGfOHHbs2MGkSZN44YUX2LVrF4sXL8bj8dC7d+8mh4EON20DUEqpMJs0aRKzZ89mzpw5XHLJJZSVlZGbm4vH4+HTTz9l8+bNLW5/8skn1w8ot2LFCpYvXx6SuPQOQCmlwuxHP/oR5eXldOvWjS5dunD55ZdzzjnnMGzYMEaPHs3gwYNb3P6GG27g6quvZsiQIQwZMoRRo0aFJC5HJQBtA1ZKOdV33x1sfO7YsSPz589vcr2KigrAmhS+bhjo5ORkZs+eHfKYtApIKaXilCYApZSKU5oAlFIxL8jxJh2vrd/TWQkgPn6HSqkQ8nq9FBcXx3wSMMZQXFyM1+sNehttBFZKxbTu3btTWFjIrl272vVz/X4/Llf7XmN7vV66dw9ugkVwWAJQSqm28ng89OnTp90/1wkT7TirCkgppVTIaAJQSqk4pQlAKaXilKMSgNFmYKWUChlHJQCllFKhowlAKaXiVDCTws8UkSIRWdFM+WARmS8iB0Tk9mC2FZFsEfm3iKyz/80KKlqtAVJKqZAJ5g7gGWBCC+UlwFTg4TZsOw342BgzAPjYfq+UUqodtZoAjDHzsE7yzZUXGWMWAtVt2PY84Fn79bPA+cEEqzcASikVOpF6EjjPGLPdfr0DyGtuRRGZAkwB8HbuR0lJs7koqvh8Po01DJwSq1PiBI01XJwQa8SHgjDGGBFp9uLeGDMDmAGQ2m2gifZHq+s44THwOhpr6DklTtBYw8UJsUaqF9BOEekCYP9bFKE4lFIqbkUqAbwFXGW/vgp4M0JxKKVU3Gq1CkhEZgFjgY4iUgjcA3gAjDFPiUhnYBGQAfhF5BYg3xizt6ltjTH/AB4CXhaRa4HNwKXBBKuNwEopFTqtJgBjzORWyncATQ5A3dy2xphiYHwwATbcsM1bKKWUaoY+CayUUnFKE4BSSsUpTQBKKRWnHJUAtAlAKaVCx1EJQCmlVOhoAlBKqTjlsASglUBKKRUqjkoAevpXSqnQcVQC0AyglFKh46wEoJRSKmQ0ASilVJzSBKCUUnFKE4BSSsUpRyUAbQNWSqnQcVQCUEopFTqaAJRSKk5pAlBKqTjluARgjLYEKKVUKDgwAUQ6AqWUig1BJQARmSkiRSKyopnywSIyX0QOiMjtjcomiMgaESkQkWkBy8eLyBIRWSoiX4hI/2Bi8WsGUEqpkAj2DuAZYEIL5SXAVODhwIUi4gYeByYC+cBkEcm3i58ELjfGjABeBO4OJhA9/SulVGgElQCMMfOwTvLNlRcZYxYC1Y2KjgUKjDEbjDFVwGzgvLrNgAz7dQdgW3CxBLOWUkqp1iSEef/dgC0B7wuB4+zX1wFzRcQH7AXGNLUDEZkCTAFI7NyfkpISEhOiv+nC5/NRUtJszowqGmvoOSVO0FjDxQmxhjsBtORW4ExjzNci8ivgEayk0IAxZgYwAyCpywCTmZWF1+Nu30gPQ0lJCdnZ2ZEOIygaa+g5JU7QWMPFCbGG+1J6K9Aj4H13YKuIdAKOMsZ8bS9/CfhxMDvUKiCllAqNcCeAhcAAEekjIonAZcBbwB6gg4gMtNc7DVgVzA6NNgMrpVRIBFUFJCKzgLFARxEpBO4BPADGmKdEpDOwCKtR1y8itwD5xpi9InIT8AHgBmYaY7639/lz4FUR8WMlhGuCicWv53+llAqJoBKAMWZyK+U7sKp3miqbC8xtYvnrwOvBfH6j7dq6iVJKqSZEf3eaRvT0r5RSoeG8BKAZQCmlQsKBCUAzgFJKhYIDE0CkI1BKqdjgvAQQ6QCUUipGOC8B6C2AUkqFhOMSgD4HoJRSoeG4BKBPAiulVGg4LgHo+V8ppULDcQlAz/9KKRUajksAOiWkUkqFhuMSgJ7/lVIqNJyXACIdgFJKxQjnJQC9BVBKqZBwYAKIdARKKRUbNAEopVSccl4C0FYApZQKCeclAD3/K6VUSDguAehzAEopFRqtJgARmSkiRSKyopnywSIyX0QOiMjtjcomiMgaESkQkWkBy0VEHhSRtSKySkSmBhuwnv6VUio0grkDeAaY0EJ5CTAVeDhwoYi4gceBiUA+MFlE8u3inwE9gMHGmCHA7GAD1hsApZQKjVYTgDFmHtZJvrnyImPMQqC6UdGxQIExZoMxpgrrJH+eXXYDcL8xxl+3j+BD1gyglFKhkBDGfXcDtgS8LwSOs1/3AyaJyAXALmCqMWZdUzsRkSnAFIDEzv3ZU1pGSULjXBN9fD4fJSXN5s2oorGGnlPiBI01XJwQazgTQEuSgEpjzGgRuRCYCZzU1IrGmBnADICkLgNMRkYHsrPT2y/Sw1RSUkJ2dnakwwiKxhp6TokTNNZwcUKs4ewFtBWrnr9Od3sZWHcDr9mvXweGB7tTfQ5AKaVCI5wJYCEwQET6iEgicBnwll32BjDOfn0KsDbYnWojsFJKhUarVUAiMgsYC3QUkULgHsADYIx5SkQ6A4uADMAvIrcA+caYvSJyE/AB4AZmGmO+t3f7EPCCiNwKVADXBRuwPgeglFKh0WoCMMZMbqV8B1b1TlNlc4G5TSwvBc4KMsZG2x7OVkoppRpz3JPASimlQsNxCUCrgJRSKjQclwD0/K+UUqHhvAQQ6QCUUipGOC8B6C2AUkqFhOMSgF/P/0opFRKOSwBaCaSUUqHhuASgNUBKKRUazksAkQ5AKaVihOMSgF8bAZRSKiQclwD09K+UUqHhvASgGUAppULCeQlA7wGUUioknJcA9PyvlFIhoQlAKaXilPMSgFYBKaVUSDgvAej5XymlQsJxCUDnA1BKqdAIKgGIyEwRKRKRFc2UDxaR+SJyQERub1Q2QUTWiEiBiExrYtvHRKQi2ID19K+UUqER7B3AM8CEFspLgKnAw4ELRcQNPA5MBPKBySKSH1A+GshqQ7yaAZRSKkSCSgDGmHlYJ/nmyouMMQuB6kZFxwIFxpgNxpgqYDZwHtQnhz8Bd7QlYG0EVkqp0Ah3G0A3YEvA+0J7GcBNwFvGmO1t2aHfH6LIlFIqziVE4kNFpCtwCTA2iHWnAFMAEjv3p7y8nJIST3gDDAGfz0dJSbM3TVFFYw09p8QJGmu4OCHWcCeArUCPgPfd7WVHA/2BAhEBSBGRAmNM/8Y7MMbMAGYAJHUZYFLT0sjOzg5z2EeupKTEEXGCxhoOTokTNNZwcUKs4U4AC4EBItIH68R/GfD/jDHfA53rVhKRiqZO/k2JxRaA376xgne/2864QblMv2gYCW7H9c5VSjlQUAlARGZhVdd0FJFC4B7AA2CMeUpEOgOLgAzALyK3APnGmL0ichPwAeAGZton/8MWi5PCP79gMwCvLinkhrF96Z+bHuGIlFLxIKgEYIyZ3Er5DqzqnabK5gJzW9k+LZg4rHWDXdOZNu7e32QCMMZgV5cppVRIOK6uIdbO/7srDjR4v3Zn+SHrvL1sG33unMvWUl97haWUigOOSwCxNhRE3Qn/klHWDdSfPljD+ysO9oytrvXzi1nfAvDxqp3tH6BSKmY5LgHE2Pmfjbv3AXDraQO579wfAXD9v5bwxGcFAKzfdXCUjL9+UtD+ASqlYpbjEkCs3QFsLt5PUoKLzhleLhp1sBnlj+9bdwL/+Hxj/bLh3TtEIkSlVIxyXAKoqY2tBFC2v5rMFA8ul5CWlMDfrhxVX3b9v5bwyuJCABJcgq+6NlJhKqVikOMSQHVtbI0Fsb+6lpTEg52xThuSR15G0iHr9e2UyoZd+2KyG6xSKjI0AUSYr6qGZI+7/r3LJXzx658cst6EH3Vme1kla5roJaSUUofDcQmgKsaqgPZX1ZKa5G6wzON20T+34aMR5x9tjaH3yeqidotNKRXbHJcAYu0OYF9VLcmJhz6Pd/0p/QDwelx8f98Z9O2UxqC8dBZujO7BpZRSzuG8BFATWwmgorKa9KRDE8CJ/TsCMP2i4aTa5YO7pLN2Z9CTpymlVIsiMhz0kYi1O4C9lTWkew/9NXTu4GXDH87E5To4/MOgzum8uXQbeyuryfBG/5DYSqno5qg7ACH22gDKK6vJSG76ZB548gcY3NkaI2jVtr2U7W88+ZpSSrWNo+4ARITKGOoLX1Xjp7La32QVUFMG5lkJYNKMBQCsfWAiiQmOyuFKqSjiqLOH2wV79ldFOoyQ+aFkP0CTVUBN6ZaZ3OD9V+t3hzwmpVT8cFQCSHAJxRWxkwCue3YhYLUDBENE+P35Q+vfX/vsItbsKGf6+6upirHGcaVU+DkqAbhFKK+MnbrvTcXWHYDbFfw4//ldMupf1/oND7y7kic/W897ASOIKqVUMByVAESE/VWx0wZwWn4eANee2CfobfK7ZHDqkFxuGmfNoPn5Oqsa6ObZSxsMI62UUq1xVAJwCTGVAGpq/Qzr1gGvx936yrbkRDd/v+oYzj+66yFlD3+4NpThKaVinMMSQGz1Aqqs9uP1HN6voF+ng0NFPDppBCcN6MjG3fvYG0NVZEqp8Gr17CMiM0WkSERWNFM+WETmi8gBEbm9UdkEEVkjIgUiMi1g+Qv28hX2/oN6qinW7gAqa2rbdPUfSESYceUobj11IOce1ZWp4wdQ6zc8+dl6lm4pDXGkSqlYFMzl5zPAhBbKS4CpwMOBC0XEDTwOTATygckikm8XvwAMBoYBycB1wQTrtsfE37MvNnoC+aoOPwEAnP6jztx86gBcLmFUzywyvAk8+dl6zn/8y5g5Rkqp8Gk1ARhj5mGd5JsrLzLGLAQa1z0cCxQYYzYYY6qA2cB59jZzjQ34BuhOEJLsh5627NkfzOpR70CN/4gSQCCXS+ielVL/fvLTC4KeO+DLDXt477vt1Pqt9f1+wwPvrGSdDj2tVEwL55PA3YAtAe8LgeMCV7Crfq4Ebm5uJyIyBZgC0KlrT1KAktIySlKiu9+7z+ejpKTlkTv37DuAV2pbXS9Y1/+4KzO/NiwtLGf1jnLmfF3A+IE5rW73izmrAfjZcV2Zekovftjj4+9fbOS1JYV8dNPokMQWKsEc12jglDhBYw0XJ8Qa6aEgngDmGWM+b24FY8wMYAbAgPxhphpITUsnOzu7nUI8PCUlJS3G+JeP1lHqqyE3Ky1k3+Xc7GzOPaYfd762nFnfbGFrhWl133WT0gM88/U2fnfeCEb+cb71HfZXk94hE487evoKtHZco4VT4gSNNVycEGs4/7K3Aj0C3ne3lwEgIvcAnYDbgt1h3eNSsTAv8P9+ZHXZTHCF/lfwhwuG0SM7mYKiloeOrqn1M+7hzxos+37b3gbvJ/6l2dyslHK4cCaAhcAAEekjIonAZcBbACJyHXAGMNkYE3RdjoiVAurqqo9UVY0/YqNqjuyZCcDPT+4b8n2LCANy01tNAOf89cv611eO6QXA3+atByAl0U1eRhIFRRVsK/WFPEalVOQF0w10FjAfGCQihSJyrYhcLyLX2+WdRaQQ60r+bnudDGNMDXAT8AGwCnjZGPO9vdungDxgvogsFZHftSXoFdvKuO7ZhUf8TMAD767kqPs/bPVEGWr/9/E6lvxQyuheWaQFORJoWw3ITWPD7n3UtDB/wqrt1tV+aqKb288YBMA7y62niV+acjzPXWM12XxRoIPOKRWLWj37GGMmt1K+g2Z68Rhj5gJzm1h+WGc9+waAh96zGi0LiioY2q1Dm/dTtLeSjmlJPDd/MwCnPvIfPrz15PrhlsPtz/+2qn/8QfbSORz9c9OoqvHz2rdbOT0/j8yUxEPWyfAmsLeyhh5ZXjo0mpOgV8cU0hITSPcmsHRLKZeO7nHI9kopZ4ue1r0gNB4yreQw+rqv21nOsX/4mFteWtpg+en/O49NAQ2i4VR3sv3LZUeH7TPqJpW/Y85yJj/9dZPrJCdaXVAfuWBQg+VZKR4yvB5cLmFAbhovfv0DFQeCG7FUKeUcjkoAjX29sbjN26zeYfVtf2vZtkPKxj78GYVBPGNQ5qvmw+93tPmzwap2KfNVc8PYfvTITml9g8M0IOBuZtX2vYdMpWmMYc++aq4/pR+dM5IAeOcXJ/L0T0fz7e9Or19vTF+rG+lfPtJxhpSKNY5KAHWNwHU27Gr7FXtpwIQyGd4EJh/bk6yUg9Uft728DIBXFm1heWEpxhje+HYre/ZV1Tc+/9fzi5jy/GJ2lR9o8+efZze8hntMo7SkBBbffWr9+7te/65B+b6qWqpq/WSnHvzuQ7t1qB+htM7U8QPI75LBy4sKY2ocJqWU0xJAo/fbyyrbvI+6WbgAumWl8IcLhrLkt6fVT7TyzcYSFmwo5ldzlnPuX7+kz51zueWlpRz9+38zdfa3ACzYYD3c8dR/1rd6x7B+V0WDAdqq2nFS+5y0JJbfezqJbhevf7uVAzXWCfxfCzbz3PxNAGQ10TYQyOtx89uz8ynzVdc3ECulYoOjEkBjS7eU8sa3W1vs6RKosrqW5xdsrn9/6pBcRAQR4coxvZj3q3EAXGbPudvYu8u3N+g2+o8vNjLt1e+aXBfgjjnLGP/n/3DWY1Zf+sC7jwuPDmr0iyOW4fXw8KVHUV1rmPnFJnpPe5e731jBH99fA0B2assJAGBM32ySPW5uf2VZ0O0uby7dyrNfbTqS0JVSYeaoBCBNTJx1y0tLeSbIE82fPlhDZbWf4d07MO9X47jttIENynvmpDT4jJX3n1H/ulO6VU9+2dMNk8MXBbv5z9pdTU7J+PKiQgC2lPjw+w1//aQAgGkTBzOse9t7Lx2u8YNzAZj+/upDyoIZi0hESLEbjI958KMW1z1QU0vvae9y8+yl3PPW9+3WsK6UajtHJYDmrGz09GpTnv1qE//4YiMA15/Szz7ZH5pRbjilHwA3jx9ASmICX037CS//1/E8eflI4GDf+ZemjOHE/h0BuGrmNwy8+70Gg68t39ZwILWV2/dSbF89XzKqfa7+66QmJXBdwKxjFxzdrf51zyAbomf8dBRgPYT37Q97ml1v3tqGzwxc+ORX9VVPSqno4qgEkOASHrn0qEOWv/bt1ibWPsjvN9zzlvUM2jUn9OHMYV2aXXeIPedulw5eALpmJnNsn2xG987m6hN61693XN8c/nXdcfVP9IJ1pV/nvvesJ2rrpvv9bE0R63dVMLpXFjlpSS3GGw7TJg6ufz11/AA2PXQWK7SFrqoAABZLSURBVO8/I+ieSKN6ZfPxL08B4IInvmL8nz/jzaVb8Qc8lb2/qoZH7Gccvv7NeJISXJTsq2LQ3e8HlaSVUu3LUQkAYNwgqzojwSUcFVCN0tJMWEsLD06QEngSb8rZw7vwwnXHNfng02/PyufCkd146oqR9cumXzS8/vUS+8p4f1UNG4utZLDuwTMZ3r0DD3+4luWFZYzsldXi54dLgtvFP68+htlTxtCnYyoAKYltex4vcBay9bv2cfPspfT9zcHn/N5auq3+DiknNZF/XXdw8NcH5648kvCVUmHguARQV2ftEuH1/z6hvmpm8+7me+Nc+MRXAMz6+ZhWr3hFhBP6d8TlOrR6yOUSHrl0BBOGHryD6J+bxu2nW20Jy+xEUzeg2sWjuuN2SYNkcjhdR0Nl3KDc+n79h+uj207hV2cMqm8TASgqr6Sm1t9gSI0Et4tjemfXt6N8WVDMmh06v4BS0cRxCSApwUXnDC9/uHAYLpfQp5N1NbuxuOnGxsAeQj1zQv/glYhw008GAPDPLzdR6zds3WNd/d8w1mpPOD+gzv3Gcf1CHkN76p+bxo3j+rPwrlP5x1XWXAHHPvgx/e96j7/bbSyBUhITeOoKq/1g0eboHhtdqXjjuATgcgkLfjOei+2G1F7ZVgJ449utXPvMwgZTIS7aVEL/u94D4N5z8umWmRy2uBLtMfML9+xnd4V1ld/RrutPS0rg9Pw8fnp8L/rnts94Q+1hRI/MQ5blZSTxzV3jGyw740d5pCUl8PgnBQ3aDABufGEJv56zPKxxKqWa5rgE0Fhyopu+HVP5ZHURH68u4rFP1vHvlTtZuW0v/2d3uwQ4vl/HsMbx0n+NAax2gGWFZWQmJ5DhPVjHPuOno7n/vKFhjaG9NW7M7pGdzPxp48lN9zZYLiIc0zuLbWWVvLH0YIO9MYZ3v9vOS4u20HvauxQUaRWRUu3J8QkArMni6/zzy038/LlFnPnY5/V18gD97KqicBncOYOc1ERuf2U5SzbvYWiXtCa7mcaa1/77x/Wvf3naoCbbTgD+Mtka+O6OOcsp81kN9iu2NuwZ9Pt3VgHwq1eW8Ut7SA6lVPjERAIIbJAMVLq/GhGYNLoHCWGe1jA50c2vJw622gBKfeQ1E1OsGdkzi00PncWq+ydw3oiuza6X4fVw58TB1PgNk2csoNZv6nsM/fHi4RzdM5P/rN1F72nv8sriQl5dUsiNLy5h1jc/tNdXUSruxEQCePSyEc2W3XbqQKZfPLzZ8lC6dHQPcuyhFXLTWx9iIZYkJ7pbveO5wp51bOX2vfzP3FX1XXcnDO3MHWcMPmT9d5dv587XvtPeQ0qFSUwkgNx0LyvuO4NV90/g2WuO5dFJBxNC3Zj37aWucTrd276f6wSpSQlcdbyVBP7+xUYeeNeq8klLTCC/awY9s1P4wwXD+OLX4xps98xXh/YuUkodufDMRxgBdVMrnjKwU30d88C8NH72497tGsekY3qwvaySCUPC2+jsVHefnU//3DR+++b39ctcLqFDsod5dxw88X902ync+dpyOiR7eGnhFi47pidHNdHrSCl1+GImAQTqkOxh00NnReSz+3ZK47HJR1NSon3em+Jxu7jy+N74qmv5w9zVfGIPL9FY/9w0Xrn+xxTtreTYP3zMm0u3MaRLRoPxlpRSRyaoKiARmSkiRSKyopnywSIyX0QOiMjtjcomiMgaESkQkWkBy/uIyNf28pdEJL4qzePclJP7semhs+gbMLxEU3IzvJyen8fMLzcy8O73uOK573jj261MeHQe099fHdQMbkqppgXbBvAMMKGF8hJgKvBw4EIRcQOPAxOBfGCyiOTbxdOB/zXG9Af2ANcGH7aKJ7efcXDO4lU793HLS0tZvaOcJz9bz6S/LWhxHCilVPOCSgDGmHlYJ/nmyouMMQuBxn+JxwIFxpgNxpgqYDZwnljdRX4CzLHXexY4v63Bq/gwMC+dWT8fw4wrR9UvG9Ejk57ZKWwt9TH83g/54PsdOmWlUm0U7jaAbsCWgPeFwHFADlBqjKkJWN6NJojIFGAKQLdu3RxTt+7z+TTWEBqUJUAiL/10CDO/KeLuM/pRawxjH1sIwH89v5iT+mXx6IWDouIBPCcc0zoaa3g4IdaobwQ2xswAZgCMGDHCZGdnRzii4JSUlKCxht4A4G8/61v//u2bTuTrjcUs3ryH91bsYPbyPZx7VNeg5zkIFycdU401PJwQa7ifA9gKBA6s391eVgxkikhCo+VKtcmw7h247qS+PP7/RpKS6OZPH6zhpD9+2uKsZUopS7gTwEJggN3jJxG4DHjLWH35PgUutte7CngzzLGoGOZyCf9z4TA8bqv65963vqdcG4eValGw3UBnAfOBQSJSKCLXisj1InK9Xd5ZRAqB24C77XUy7Dr+m4APgFXAy8aYuieAfg3cJiIFWG0C/wjtV1Px5rwR3Vj7wESmXzSMZYVlTPrbAraV+lrfUKk4FVQbgDFmcivlO7CqcZoqmwvMbWL5BqxeQkqFjIgw6ZieJCa4uPWlZdzwwhLevPGESIelVFSKibGAlGrsgqO7c+O4fizbUqrtAUo1QxOAilnXntgXj1u44ImvGPunT+vHiFJKWTQBqJiVnZrIKQM7AbCpeD9n/uVznvxsPcX2lJ1KxTtNACqmPXjBMK4/pR+3nDqAMl81099fzagHPuLd5dupqvFHOjylIirqHwRT6kjkZXiZNtGabOamcf256p/f8GVBMTe+uASAuVNPYmBeWthnjFMqGun/ehU3EtwuXrhuDKt/P4FjemcBcOZjn3P+E1/i9x86zPTizSV6l6BimiYAFXe8HjevXP9jZv5sNBneBFZs3ctZ//cF28t8bC318f22Mgr37OeiJ+cz9J4PKCiqiHTISoWFVgGpuPWTwXks+e1pXPGPr1mwoYTj/+eTQ9apqvVz+v/+h/FD8shJTaRwj4/NJfvYUmI9YNYxLYmbxvXj8jG9qPUbNhfvp1fOoeMQ7TtQQ2qS/rmp6KL/I1VcS3C7mD3leD5auZPrnlsEwIDcNNbZV/1v33Qiby7dynMLNtdXB6V7E/C4hepaw+6KA9z79krufXtl/T4zvAmM6JbO+uJKTsvPo2d2Cve/sxKXwA1j+zFxaBdSEt2tTobTHGMM1bWGxAS9gVdHRpw0xd6IESPM0qVLIx1GUJwwEmAdjdXi9xuq/X4S3S7ue3sl/XPTuGKMNYl9xYEaElxCUoKrfrhpYwwHavzM/HIjf3x/DZ0zvJxzVBfW7qzg+62lZKUm1SeSOi6BuuaGE/t35IyhnRndK4vuWckke9wkuF3U1PpJcLso2lvJV+uL8Xrc7K+q4dsfSrloVHfOf/xLALplJjP52B7899j+uFxCrd+wv6qGtKSENg2Jrb//8IimWEVksTFm9CHLNQGERzT98lujsR45Y0yDk25dnNPfX82aHeVMv2g4ndKT2FKyny8KdrNmRzlvLdtGyb6qBvs5tnc232wqYXj3DiwvLGv284Z2y2DPvmq22mMdXTmmF88v2FxfPrJnJtmpSWzYVUHH9CTGD87l0tE9yEo9dObVZeu3sZ8kavx+vijYTYbXQ8/sFH7cL4ectKT69baU7GfV9r10SPbQNTO5ftm+qlp27K2kdF8VR/XIpG+nVPIyvHja0LNqV/kBDIac1CTcruaTV7T+/psSTbFqAmhn0fTLb43GGnrBxFlxoIYPVuzg4Q/XsL2skmHdOvDd1oYn/eP6ZNM1MxljDIV7fCz+YQ93ThzMlJP7Ues3/M/cVfz9i42H7Ltvx1Q27N53yPL8LhlcPqYnQ7pk8F1hGbO++YHVO8rry0Wg7pTgEuiamUzXzGQ2F+9j597gH6DzuIWUxATG9M3m9PzO/PKVZQzpksEdEwYxblAutX7Dgg3FPPrRWraVVtYnMhHITkkkL8PLiJ6ZnDKwE6cM7MS8tbvYsbeSzUWlHNU7l6oaP53Sk9i4q4K8DC/lB2oor6yhb8dU9lXV4KuqpW+nVHrlpJKVkthiUmmO32/YWV5Jma+a3HQvqUlukhLch6yzr6qG1MQEXI0+I5r+r2oCaGfR9MtvjcYaem2J80BNLYKQmOBiW6mPTcX72LOvmlG9sujcwdtg3epa/yFX1ku3lHL+418yblAnHpt8dH1VkjEGv4HNxft4b8UOlmzewzcbSyg/UFO/bV5GEqcOzKZHpw70yk7hpIGdcIuwdmc5H6/ayRcFu9lbWUN1rZ/UxARuP2MgiW43W0v3U15ZQ9fMZLpnJdMpPYlEt4vVO8rZUrKfdUUVvLxoC+WVNTQ2okcm328ro7rWOveM6ZvNmL45ZKcmsrv8ALsqqigoKmd5YRkHQtANN92bwOheWXg9braVVTKsWwYjemSRmeyhZ04KeRle9vqqKfNVs2JrGV+tL+azNUXsbSL2bpnJnDSgIwVFFfxQsp8DNX7KfNUkuITs1EQM1t2gSwQXhlSvh9SkBBLdLlwiJCe6yUj20Cs7hcwUD4kJLhLdLrweNy6X4HEJVbV+KqtrcYmwt7KGXeUHyEhOoKrGT16Gl5REN2W+aioO1OBxuXC7BI9bcLtcJCW42LO/ij37q8hOTWJgXhpJCW6O65ujCaA9OeVEBRprOLR3nJXVtfaJoOVql4oDNbz33Xa+Wl/McX2yuWhUd8rLSsMSa2V1LWt3lvPRqiJOG5JHksfFP7/cxNzvttMh2UPfTqlcc0IfTraH62jMV1XL5+t2Mf391Qzvnsmlo3vw7YYdfFxQxk3j+uM3hrwMLzV+Q7o3gZREN5uL95OZ4sGb4GbD7gp+KN7P6h3lfLPRmpqxY1oS327ZU5986gS2zYB1B3Nafh5H98iiQ4qH7aWVACwvLOWbjSVkpSYysmcmXo+bHtkpVByoYc++KkSsEWmNMVTsr8QvCeyrqqGqxk+t31BZXUupr5rCPT5qm3j2pCluu33nSGyefrYmgPbklBMVaKzh4JQ4If5i3Xeghu1lPnbuPcC/V+5keWEpXTKT6ZOTyrjBufiNYWBuOh1SPGGLtbrWj6+6luoaPwdq/OyvqgWs3l0et3UlX+s3ZKZ46JDsocxXTWpSAjvKKtlbWU1mSiLZKYnU+K3EUmMnl4oDNXTJSCYlyc2efVVs3L2P6lrDSQM7NZkAtBuoUiqupCYl0D83nf656ZzQv2NEYvC4XW1qJM9MsRrvD53r2n3oyrbcDC+5Gd5my0GfBFZKqbilCUAppeKUJgCllIpTwU4KP1NEikRkRTPlIiKPiUiBiCwXkZEBZdNFZIX9Mylg+XgRWSIiS0XkCxHpf+RfRymlVLCCvQN4BpjQQvlEYID9MwV4EkBEzgJGAiOA44DbRSTD3uZJ4HJjzAjgReDutgavlFLq8AWVAIwx84CSFlY5D3jOWBYAmSLSBcgH5hljaowx+4DlHEwkBqhLBh2AbYfzBZRSSh2eUHUD7QZsCXhfaC9bBtwjIn8GUoBxQN2widcBc0XEB+wFxjS1YxGZgnVXQbdu3SgpaSkPRQ+fz6exhoFTYnVKnKCxhosTYg3rcwDGmA9F5BjgK2AXMB+otYtvBc40xnwtIr8CHsFKCo33MQOYAdaDYPH0wEp70VhDzylxgsYaLk6INVQJYCvQI+B9d3sZxpgHgQcBRORFYK2IdAKOMsZ8ba//EvB+ax+ybNmyChFZE6KYw60jsDvSQQRJYw09p8QJGmu4RFOsvZpaGKoE8BZwk4jMxmrsLTPGbBcRN5BpjCkWkeHAcOBDe5sOIjLQGLMWOA1YFcTnrGnqceZoJCKLNNbQc0qsTokTNNZwcUKsQSUAEZkFjAU6ikghcA/gATDGPAXMBc4ECoD9wNX2ph7gc3uc9L3AFcaYGnufPwdeFRE/sAe4JjRfSSmlVDCCSgDGmMmtlBvgxiaWV2L1BGpqm9eB14P5fKWUUqHntCeBZ0Q6gDbQWMPDKbE6JU7QWMMl6mN11HDQSimlQsdpdwBKKaVCRBOAUkrFKUckABGZICJr7MHmpkVBPD1E5FMRWSki34vIzfbybBH5t4iss//Nspc3O1heO8bsFpFvReQd+30fEfnajuklEUm0lyfZ7wvs8t7tHGemiMwRkdUiskpEjo/W4yoit9q//xUiMktEvNFyXJsawPFwjqOIXGWvv05ErmrHWP9k/x9YLiKvi0hmQNmddqxrROSMgOVhP080FWtA2S9FxIhIR/t9RI9rUIwxUf2DNeXNeqAvkIg1vER+hGPqAoy0X6cDa7F6O/0RmGYvnwZMt1+fCbwHCNaQF19HIObbsAbde8d+/zJwmf36KeAG+/V/A0/Zry8DXmrnOJ8FrrNfJwKZ0XhcsYY62QgkBxzPn0XLcQVOxhqIcUXAsjYdRyAb2GD/m2W/zmqnWE8HEuzX0wNizbfPAUlAH/vc4G6v80RTsdrLewAfAJuBjtFwXIP6PpH40DYe8OOBDwLe3wncGem4GsX4JtbDbGuALvayLlgPrgH8DZgcsH79eu0UX3fgY+AnwDv2f8jdAX9g9cfY/k98vP06wV5P2inODvZJVRotj7rjysHxr7Lt4/QOcEY0HVegd6OTapuOIzAZ+FvA8gbrhTPWRmUXAC/Yrxv8/dcd1/Y8TzQVKzAHOArYxMEEEPHj2tqPE6qAmhtoLirYt/JHA18DecaY7XbRDiDPfh3p7/AocAfgt9/nAKXGfiivUTz1sdrlZfb67aEP1phR/7Srq/4uIqlE4XE1xmwFHgZ+ALZjHafFROdxrdPW4xjp/7d1rsG6koYojFVEzgO2GmOWNSqKulgbc0ICiFoikga8CtxijNkbWGas1B7xPrYicjZQZIxZHOlYgpCAdXv9pDHmaGAfVlVFvSg6rllYw6D3AboCqbQ8Z0ZUiZbj2BoRuQuoAV6IdCxNEZEU4DfA7yIdy+FwQgJodqC5SBIRD9bJ/wVjzGv24p1izYOA/W+RvTyS3+EE4FwR2QTMxqoG+gvWnA11T4IHxlMfq13eAShup1gLgUJzcJDAOVgJIRqP66nARmPMLmNMNfAa1rGOxuNap63HMaJ/eyLyM+BsrImj6pJVtMXaD+siYJn9N9YdWCIinaMw1kM4IQEsBAbYvSsSsRrQ3opkQCIiwD+AVcaYRwKK3gLqWvSvwmobqFv+U7tXwBjswfLaI1ZjzJ3GmO7GmN5Yx+4TY8zlwKfAxc3EWvcdLrbXb5crRWPMDmCLiAyyF43Hmj8i6o4rVtXPGBFJsf8/1MUadcc1QFuP4wfA6SKSZd/xnG4vCzsRmYBVbXmuMWZ/o+9wmd2rqg/WLITfEKHzhDHmO2NMrjGmt/03VojVQWQHUXhcm/oCUf+D1Zq+FquV/64oiOdErNvn5cBS++dMrDrdj4F1wEdAtr2+AI/b8X8HjI5Q3GM52AuoL9YfTgHwCpBkL/fa7wvs8r7tHOMIYJF9bN/A6iURlccVuA9YDawAnsfqmRIVxxWYhdU2UY11Urr2cI4jVv17gf1zdTvGWoBVT1739/VUwPp32bGuASYGLA/7eaKpWBuVb+JgI3BEj2swPzoUhFJKxSknVAEppZQKA00ASikVpzQBKKVUnNIEoJRScUoTgFJKxSlNAEopFac0ASilVJz6/wBNE632vEpFAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "not enough values to plot a chart\n",
      "Total time        : 0:46:37.044966\n"
     ]
    }
   ],
   "source": [
    "tfms = [None, TSClassification()]\n",
    "batch_tfms = TSStandardize(by_sample=True)\n",
    "dls = get_ts_dls(X_large_zarr, y_large_zarr, splits=splits, tfms=tfms, batch_tfms=batch_tfms, inplace=False, bs=[512, 1_024], num_workers=0)\n",
    "learn = ts_learner(dls, InceptionTimePlus, metrics=accuracy, cbs=ShowGraph())\n",
    "timer.start()\n",
    "learn.fit_one_cycle(1, 1e-2)\n",
    "timer.stop()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>accuracy</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1.098617</td>\n",
       "      <td>1.098628</td>\n",
       "      <td>0.332385</td>\n",
       "      <td>26:06</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEXCAYAAACkpJNEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZQc5Xnv8e/TPT2bFjSL1pFAwmCQQDCsxjEXC+MFsM2SgIEQTAIcObYJ4OXG2M4BHJxzsYOX+MQgC5tgEkAmLMHhgrFN4Mo+LEZgIQQSCCRhLWhBrWUk9Szd/dw/qnrUGmY0LU3NdNfM73NOn65636rqp0uaevp931rM3RERkZEnUe4ARESkPJQARERGKCUAEZERSglARGSEUgIQERmhlABEREYoJQARkRFKCUBGLDNbbWYfLXccIuWiBCAiMkIpAYgUMbMaM/uhma0PXz80s5qwrtnMHjWzbWaWNrPfmVkirPuama0zszYze93MzgjLE2Z2vZm9ZWZbzOx+M2sM62rN7D/C8m1m9oKZTSzft5eRRglAZG/fBE4BWoFjgZOBfwjrvgKsBcYDE4FvAG5mRwBXAye5+xjgE8DqcJ2/A84DPgxMAbYCPw7rLgcOAqYBTcDfApnB+2oie1MCENnbpcA/uvsmd98MfAu4LKzrAiYDh7h7l7v/zoObaeWAGmCWmaXcfbW7vxWu87fAN919rbt3ADcBF5hZVbi9JuAwd8+5+4vuvmPIvqmMeEoAInubArxdNP92WAbwz8CbwK/NbKWZXQ/g7m8C1xEc3DeZ2QIzK6xzCPBw2MWzDVhGkDAmAv8OPAEsCLubvmtmqcH9eiJ7KAGI7G09wUG74OCwDHdvc/evuPuhwDnAlwt9/e5+r7ufGq7rwHfC9dcAZ7n7uKJXrbuvC1sR33L3WcCfAZ8CPjsk31IEJQCRVDgYW2tmtcB9wD+Y2XgzawZuAP4DwMw+ZWaHmZkB2wl+yefN7Agz+0g4WNxO0I+fD7c/D/gnMzsk3MZ4Mzs3nD7dzGabWRLYQdAllEdkiCgByEj3GMEBu/CqBRYBS4BXgJeAb4fLHg78FtgJPAvc5u5PEfT/3wK8C2wAJgBfD9f5F+CXBN1GbcBzwAfCuknAAwQH/2XA/yPoFhIZEqYHwoiIjExqAYiIjFBKACIiI5QSgIjICKUEICIyQikBiAyAmU03Mw+v7BWJFSUAkTIKrwV4ysy2m9nqPpb5oJk9E07fbGavmFnWzG4aylhl+FECECmvXcCdwP/exzKfJLheAYJbUfw98H8HOS4ZAZQAZFgxsylm9qCZbTazVWZ2TVHdTWb2gJn9Irxt80tmdmxR/Uwzezq8b8+rZnZOUV2dmX3PzN4Of63/3szqij76UjP7k5m9a2bfLDVed/+Du/87sHIfi51NmADc/efu/jjQVupniPRFCUCGjfDe/P8NvAy0AGcA15nZJ4oWOxf4T6ARuBf4LzNLhTdh+2/g1wRX8v4dcE94q2eAW4ETCO7Z00jwK7z4tg2nAkeEn3mDmc0MYzo1vAncgX6nyQQ3jvvjgW5DpC9KADKcnASMd/d/dPdOd18J3AFcXLTMi+7+gLt3Ad8nuPXDKeFrNHBLuO7/AI8Cl4SJ5Qrg2vAmbjl3fya8vXPBt9w94+4vEySgYwHc/ffuPm4A3+ls4FeuS/ZlEOjMBRlODgGm9PjFnQR+VzS/pjDh7nkzW8ue2z2vcffiX/VvE7QkmgkSxVv0bUPR9G6CZBKFswlaKiKRUwKQ4WQNsMrdD9/HMtMKE+Ev+6mEt3sGpplZoigJHAy8QXCTt3bgfQS/7odE2C31YeBvhuozZWRRF5AMJ38A2sLn89aZWdLMjjazk4qWOcHM/jw8b/86oIPgDp3PE/xy//twTGAO8GlgQZgQ7gS+Hw4yJ8NTM2sGGnD4zOBaIBXMWq2ZVYfVpwJLip8SFsZWS/C3WxUunxxoHDIyKQHIsOHuOYKHqrQCqwh+uf+U4Lm7BY8AFxE8m/cy4M/DB7N0EhzwzwrXuw34rLsvD9f7KsHtoV8A0gQPfOn378fM/peZ7dzHIqcR3Ib6MYIWR4ZgIBr2Pv2z4I5wmUsInl+cYc8jK0X2i24HLSNGeOHUYe7+V+WOpRRm9hpwgbu/Vu5YZHhSC0CkAoXdQHfr4C+DSYPAIhUo7JK6pdxxyPCmLiARkRFKXUAiIiNUrLqAmpqafMaMGeUOoyS5XI5kMh5n5ynW6MUlTlCsg6WSYn3xxRffdffxPctjlQCmTZvGokWLyh1GSdLpNI2NjeUOoySKNXpxiRMU62CppFjN7O3eyks5j/lOM9tkZkv7qL/UzJaE9yh/pnB3RTM7wswWF712mNl1Yd1NZrauqO7sgXw5ERHZf6W0AO4C/hW4u4/6VcCH3X2rmZ0FzAc+4O6vE1yQQ3il4jrg4aL1fuDutx5o4CIiMjD9JgB3X2hm0/dR/0zR7HME91bp6QzgLXfvtRkiIiJDL+oxgCuBx3spvxi4r0fZ1Wb2WWAR8BV339rbBs1sLjAXoKWlhXQ6HWG4gyeTySjWQRCXWOMSJwz/WHO5HFu3bqWrq2uQoupdPp/nnXfeGdLPTKVSNDQ0lDz4XNJ1AGEL4FF3P3ofy5xOcP+UU919S1F5NcHdFo9y941h2USC+604cDMw2d2v6C+O1tZWX7x4cb/xVoJKGgDqj2KNXlzihOEf66pVqxgzZgxNTU2Y2SBF9l7ZbJaqqqE7z8bd2bJlC21tbfQ8W9LMXnT3E3uuE8l1AGZ2DMFNt84tPviHzgJeKhz8w0A3hg/VyBPc3OrkKOIQEempvb19yA/+5WBmNDU10d7eXvI6A04AZnYw8BBwmbu/0csil9Cj+yd8zF3B+UCvZxiJiERhuB/8C/b3e/bbPjGz+4A5QHP49KQbCe5djrvPA24AmoDbwg/PFpoaZjYK+BjwuR6b/a6ZtRJ0Aa3upb5XuzpzpSwmIiIlKOUsoEv6qb8KuKqPul0EyaFn+QHdv/xPW0tv2oiIVIJt27Zx77338oUvfGG/1jv77LO59957GTduII+U3jfdC0hEZBBt27aN22677T3l2Wx2n+s99thjg3rwh5jdCiIxQvrxRGT4uP7663nrrbdobW0llUpRW1tLQ0MDy5cv54033uC8885jzZo1tLe3c+211zJ37lwApk+fzqJFi9i5cydnnXUWp556Ks888wwtLS088sgj1NXVDTi2WCUAEZGB+NZ/v8pr63f0v+B+mDVlLDd++qg+62+55RaWLl3K4sWLefrpp/nkJz/J0qVLu0/VvPPOO2lsbCSTyXDSSSfxF3/xFzQ17d1zvmLFCu677z7uuOMOPvOZz/Dggw/yV3818AfbxSwB6NkFIhJvJ5988l7n6f/oRz/i4YeDu+SsWbOGFStWvCcBzJgxg9bWVgBOOOEEVq9eHUkssUoAOvyLyEDs65f6UBk1alT39NNPP81vf/tbnn32Werr65kzZ06v5/HX1NR0TyeTSTKZTCSxxGsQWBlARGJmzJgxtLW19Vq3fft2GhoaqK+vZ/ny5Tz33HNDGlusWgAiInHT1NTEhz70IY4++mjq6uqYOHFid92ZZ57JvHnzmDlzJkcccQSnnHLKkMYWqwSgBoCIxNG9997ba3lNTQ2PP97b/TPp7udvbm5m6dI9N0v46le/Gllc8eoCEhGRyMQuAZRy91IREelf7BJAXsd/EZFIxDABKAOIiERBCUBEZISKXQLQ8V9EJBqxSwBqAYjIcDZ69GgA1q9fzwUXXNDrMnPmzGHRokUD/qzYJYCcRoFFZASYMmUKDzzwwKB+RuwSgI7/IhIn119/PT/+8Y+752+66Sa+/e1vc8YZZ3D88ccze/ZsHnnkkfest3r1ao4++mgAMpkMF198MTNnzuT888+P7F5AsboSGHQdgIgMwOPXw4ZXot3mpNlw1i19Vl900UVcd911fPGLXwTg/vvv54knnuCaa65h7NixvPvuu5xyyimcc845fT7T9/bbb6e+vp5ly5axZMkSjj/++EhCj10CUAtAROLkuOOOY9OmTaxfv57NmzfT0NDApEmT+NKXvsTChQtJJBKsW7eOjRs3MmnSpF63sXDhQq655hoAjjnmGI455phIYothAlAGEJEDtI9f6oPpwgsv5IEHHmDDhg1cdNFF3HPPPWzevJkXX3yRVCrF9OnTe70N9GDrdwzAzO40s01mtrSP+kvNbImZvWJmz5jZsUV1q8PyxWa2qKi80cx+Y2YrwveGUgPOqwkgIjFz0UUXsWDBAh544AEuvPBCtm/fzoQJE0ilUjz11FO8/fbb+1z/tNNO676h3NKlS1myZEkkcZUyCHwXcOY+6lcBH3b32cDNwPwe9ae7e6u7n1hUdj3wpLsfDjwZzpdEx38RiZujjjqKtrY2WlpamDx5MpdeeimLFi1i9uzZ3H333Rx55JH7XP/zn/88O3fuZObMmdxwww2ccMIJkcTVbxeQuy80s+n7qH+maPY5YGoJn3suMCec/jnwNPC1EtZTF5CIxNIrr+wZfG5ububZZ5/tdbmdO3cCwUPhC7eBrqurY8GCBZHHFPUYwJVA8c2tHfi1mTnwE3cvtA4muvs74fQGYCJ9MLO5wFyA6kmHkd66jdp8NKdADaZMJkM6nS53GCVRrNGLS5ww/GPN5/Nks9lBiqgyP7fUfRRZAjCz0wkSwKlFxae6+zozmwD8xsyWu/vC4vXc3cME0aswacwHqJl8uI8eM5bGxlF9LV4x0uk0jY2N5Q6jJIo1enGJE4Z/rBs3bqSqaujPd8lms2X53EQiUfI+iuRCMDM7BvgpcK67bymUu/u68H0T8DBwcli10cwmh+tOBjaV+lnZfD6KkEVkBBkp1w/t7/cccAIws4OBh4DL3P2NovJRZjamMA18HCicSfRL4PJw+nLgvZfB9aEjqwQgIqWrra1ly5Ytwz4JuDtbtmyhtra25HX6bZ+Y2X0EA7bNZrYWuBFIhR84D7gBaAJuC69iy4Zn/EwEHg7LqoB73f1X4WZvAe43syuBt4HPlBpwV254/yOKSLSmTp3K2rVr2bx585B+bj6fJ5EY2rvt1NbWMnVqKefhBEo5C+iSfuqvAq7qpXwlcOx714Cwm+iMEmPcS1dOLQARKV0qlWLGjBlD/rlxGFuJ3c3gOtUFJCISifglALUAREQiEbsE0KUWgIhIJGKXANQCEBGJRuwSgJ4IJiISDSUAEZERSglARGSEil0C0N1ARUSiEbsEoDFgEZFoxC8BqAUgIhKJ2CUAPRJSRCQasUsAGgQWEYlG7BKABoFFRKIRuwSQVQtARCQSsUsA6gISEYlG7BKABoFFRKIRuwSg00BFRKIRuwSgFoCISDRilwDUAhARiUa/CcDM7jSzTWa2tI/6S81siZm9YmbPmNmxYfk0M3vKzF4zs1fN7NqidW4ys3Vmtjh8nV1KsIZuBSEiEpVSWgB3AWfuo34V8GF3nw3cDMwPy7PAV9x9FnAK8EUzm1W03g/cvTV8PVZqwLoOQEQkGv0mAHdfCKT3Uf+Mu28NZ58Dpobl77j7S+F0G7AMaBlIsGY6DVREJCpVEW/vSuDxnoVmNh04Dni+qPhqM/sssIigpbC153rhunOBuQA1k97H7kyGdLrPfFQxMjGJExTrYIhLnKBYB0scYo0sAZjZ6QQJ4NQe5aOBB4Hr3H1HWHw7QXeRh+/fA67obbvuPp+wW6l+yvs9VV1DY2NjVGEPmnQ6HYs4QbEOhrjECYp1sMQh1kjOAjKzY4CfAue6+5ai8hTBwf8ed3+oUO7uG9095+554A7g5NI+SF1AIiJRGXACMLODgYeAy9z9jaJyA34GLHP37/dYZ3LR7PlAr2cYveez0CCwiEhU+u0CMrP7gDlAs5mtBW4EUgDuPg+4AWgCbguO+WTd/UTgQ8BlwCtmtjjc3DfCM36+a2atBF1Aq4HPlRqwWgAiItHoNwG4+yX91F8FXNVL+e8JfrT3ts5lpQZYTNcBiIhEJ15XApu6gEREohKrBGCYuoBERCISqwQAuheQiEhUYpUADN0NVEQkKrFKAJgeCSkiEpVYJQC1AEREohOrBAAaAxARiUqsEoDuBioiEp1YJQAwXQcgIhKRWCWA4EpgJQARkSjEKgEA5HUrCBGRSMQqAZhpEFhEJCqxSgCgLiARkajEKgHoeQAiItGJVQLQE8FERKITqwSgs4BERKITqwSg6wBERKITqwSgFoCISHRilQCCJ4KVOwgRkeEhVglALQARkeiUlADM7E4z22RmS/uov9TMlpjZK2b2jJkdW1R3ppm9bmZvmtn1ReUzzOz5sPwXZlZdSixKACIi0Si1BXAXcOY+6lcBH3b32cDNwHwAM0sCPwbOAmYBl5jZrHCd7wA/cPfDgK3Alf0FYXoovIhIZEpKAO6+EEjvo/4Zd98azj4HTA2nTwbedPeV7t4JLADONTMDPgI8EC73c+C8UmJRC0BEJBpVg7DNK4HHw+kWYE1R3VrgA0ATsM3ds0XlLb1tzMzmAnMBxk6eTlcuRzrdZy6qGJlMJhZxgmIdDHGJExTrYIlDrJEmADM7nSABnBrVNt19PmGX0qT3zXLHaGxsjGrzgyadTsciTlCsgyEucYJiHSxxiDWys4DM7Bjgp8C57r4lLF4HTCtabGpYtgUYZ2ZVPcr3/RmoC0hEJCqRJAAzOxh4CLjM3d8oqnoBODw846cauBj4pbs78BRwQbjc5cAjpXyWHgovIhKNkrqAzOw+YA7QbGZrgRuBFIC7zwNuIOjXvy0Y3yXr7ie6e9bMrgaeAJLAne7+arjZrwELzOzbwB+Bn/UbB3oegIhIVEpKAO5+ST/1VwFX9VH3GPBYL+UrCc4SKp3piWAiIlGJ35XAagGIiEQiXgnAjFzeNQ4gIhKBeCWA8L1L/UAiIgMWrwQQZoCunFoAIiIDFa8EEL53ZdUCEBEZqFglgEIToDOnBCAiMlCxSgCFFkCnWgAiIgMWqwSQ6B4DUAIQERmoWCUAC9sAGgQWERm4eCUAtQBERCITywSgQWARkYGLVwII33UaqIjIwMUrAZjGAEREohKvBBC+awxARGTg4pUANAYgIhKZeCWA8F0tABGRgYtVAqB7DEAJQERkoGKVALqvA8hqEFhEZKDilQDCd40BiIgMXLwSgK4EFhGJTL8JwMzuNLNNZra0j/ojzexZM+sws68WlR9hZouLXjvM7Lqw7iYzW1dUd3Ypwe65F5ASgIjIQFWVsMxdwL8Cd/dRnwauAc4rLnT314FWADNLAuuAh4sW+YG737o/weqJYCIi0em3BeDuCwkO8n3Vb3L3F4CufWzmDOAtd397/0PcQ88DEBGJTiktgChcDNzXo+xqM/sssAj4irtv7W1FM5sLzAVoaWmhNmHs2LWbdLrPnFQRMplMxcdYoFijF5c4QbEOljjEOugJwMyqgXOArxcV3w7cDHj4/j3git7Wd/f5wHyA1tZW76xKkKyqprGxcVDjHqh0Ol3xMRYo1ujFJU5QrIMlDrEOxVlAZwEvufvGQoG7b3T3nLvngTuAk0vdWCqZ0CCwiEgEhiIBXEKP7h8zm1w0ez7Q6xlGvUklE3RqEFhEZMD67QIys/uAOUCzma0FbgRSAO4+z8wmEfTjjwXy4ames9x9h5mNAj4GfK7HZr9rZq0EXUCre6nvU3XS1AIQEYlAvwnA3S/pp34DMLWPul1AUy/ll5UaYE+pKnUBiYhEIVZXAoPGAEREohK7BFCdTNCpm8GJiAxY7BKAuoBERKIRuwSgQWARkWjELgFoDEBEJBqxTAC6DkBEZOBimQC6dDM4EZEBi10CqK7SGICISBRilwA0BiAiEo2YJgCNAYiIDFQsE4AeCi8iMnCxSwC6DkBEJBqxSwA6C0hEJBrxSwBVGgMQEYlC/BJAOAbgriQgIjIQsUsA1UkDUCtARGSAYpcAUskgZJ0JJCIyMLFLADVVYQLQQLCIyIDELgHUVwdPsdzdmS1zJCIi8Ra7BFBbnQSgvStX5khEROKt3wRgZnea2SYzW9pH/ZFm9qyZdZjZV3vUrTazV8xssZktKipvNLPfmNmK8L2h1IDrU0EC2N2pBCAiMhCltADuAs7cR30auAa4tY/609291d1PLCq7HnjS3Q8HngznS1IXtgAySgAiIgPSbwJw94UEB/m+6je5+wtA13587rnAz8PpnwPnlbpiIQHsVheQiMiAVA3y9h34tZk58BN3nx+WT3T3d8LpDcDEvjZgZnOBuQAtLS10ZXYC8G56O+n0YId/4DKZDOl0n3mzoijW6MUlTlCsgyUOsQ72EfRUd19nZhOA35jZ8rBF0c3dPUwQvQqTxnyA1tZWn9TcCECiuo7GxsZBDH1g0ul0RcdXTLFGLy5xgmIdLHGIdVDPAnL3deH7JuBh4OSwaqOZTQYI3zeVuk11AYmIRGPQEoCZjTKzMYVp4ONA4UyiXwKXh9OXA4+Uut268Cygdg0Ci4gMSL9dQGZ2HzAHaDaztcCNQArA3eeZ2SRgETAWyJvZdcAsoBl42MwKn3Ovu/8q3OwtwP1mdiXwNvCZUgOur67CDNra92fMWUREeuo3Abj7Jf3UbwCm9lK1Azi2j3W2AGeUEmBPyYTRNKqazTs7DmR1EREJxe5KYIDxY2rZ3KYEICIyELFMAOPqUmzPqAtIRGQgYpkAxtZVsSOjm8GJiAxEPBNAbYodGgQWERmQeCaAuhQ71AUkIjIg8UwAtSl2debI6qlgIiIHLJ4JoC44e7WtXeMAIiIHKpYJoKG+GoD07s4yRyIiEl+xTACTD6oFYP22TJkjERGJr1gmgCnj6gAlABGRgYhlAph0UC0Jg3VblQBERA5ULBNAKplg0tha1qoFICJywGKZAABaGurUAhARGYD4JoBxdaxVAhAROWDxTQANdWzY0a6LwUREDlB8E8C4enJ5Z6NuCy0ickBimwCmNQangr69ZVeZIxERiafYJoAZzaMAWP3u7jJHIiIST7FNAFMOqqOmKsHKzTvLHYqISCz1mwDM7E4z22RmS/uoP9LMnjWzDjP7alH5NDN7ysxeM7NXzezaorqbzGydmS0OX2fvd+AJY0bzKFa9qy4gEZEDUUoL4C7gzH3Up4FrgFt7lGeBr7j7LOAU4ItmNquo/gfu3hq+HtuPmLspAYiIHLh+E4C7LyQ4yPdVv8ndXwC6epS/4+4vhdNtwDKgZWDh7m1G8yj+lN5Nl04FFRHZb0MyBmBm04HjgOeLiq82syVhF1PDgWz30PGjyeZdF4SJiByAqsH+ADMbDTwIXOfuO8Li24GbAQ/fvwdc0cf6c4G5AC0tLaTTexojTdU5AJas2sBBiQPKIYMmk8nsFWslU6zRi0ucoFgHSxxiHdQEYGYpgoP/Pe7+UKHc3TcWLXMH8Ghf23D3+cB8gNbWVm9sbOyua60ZDSxlc7tRXF4J0ul0xcXUF8UavbjECYp1sMQh1kHrAjIzA34GLHP37/eom1w0ez7Q6xlG/WkYVc24+hQrNRAsIrLf+m0BmNl9wByg2czWAjcCKQB3n2dmk4BFwFggb2bXAbOAY4DLgFfMbHG4uW+EZ/x818xaCbqAVgOfO9AvcGjzKFZtVgIQEdlf/SYAd7+kn/oNwNReqn4PWB/rXFZSdCWY0TyahSs24+4EjQ4RESlFbK8ELjh22kFsbuvQmUAiIvsp9gngpOnBIMsLqyt7tF1EpNLEPgEcMXEM4+pT/P7Nd8sdiohIrMQ+ASQSxkeOnMCTyzbpimARkf0Q+wQA8ImjJrE908XzK9UNJCJSqmGRAE47fDx1qSS/evWdcociIhIbwyIB1FUnmXPEeH796kbyeS93OCIisTAsEgDAmUdPYlNbB39cs7XcoYiIxMKwSQCnHzmB2lSCmx9dRntXrtzhiIhUvGGTAMbWpvjuBceyeM02/uO5t8sdjohIxRs2CQDgnGOn8L8Ob+a2p99iV0e23OGIiFS0YZUAAL70sfeT3tXJXc+sLncoIiIVbdglgOMPbuDjsybyoydX8OamtnKHIyJSsYZdAgD49vlHU1+d5Mv3v6yrg0VE+jAsE8CEMbX80/mzWbJ2O197cAnuujZARKSnYZkAAM6ePZnrPno4D720jv/z+HIlARGRHgb9ofDldO0Zh5Pe1cn8hStZk97NN86eybTG+nKHJSJSEYZ1AjAzvnXOUUwYU8OPn3qLx5du4KTpDXzkyIn8+fEtTBxbW+4QRUTKZlgnAAiSwNUfOZxPHzuFaxYsZvWW3XznV8v5zq+W89GZEzhpeiNTxtVx3MHjmNqwp3WwYXs7zaOrqUoO214yERnhhn0CKDikaRSPfPFDALy+oY1/emwZC1e8y2+XbQLADJpGVWNm7OrIsrszx9jaKk6a3sicIyewqyPLYeNHA3DK+5oYXdP7rlu/LcPjSzfw/oYks6pH0TS6JrLvsG5bhufe2kJV0pgwppaJY2sYW5eiuipBdTJBbSoJwPbdXXRkc7R35amrTtI0qppEQs9LFpG9lZQAzOxO4FPAJnc/upf6I4F/A44HvunutxbVnQn8C5AEfurut4TlM4AFQBPwInCZu3cO7OuU5ohJY7j7ipNp78qxbluGTGeOX7+2kedXbgFgdE0VLQ11ZDpzPPLyep5cvmmv9RMGh08Yw5GTxzCqporXN7TRkc1hGK+u386eG5K+wpGTxnDBCVP56MyJNNRXk0jAmNrUfsX7P8s3cuMvX2VNet/PPZ5yUC1j61Is3/De6x+ObhlLbVWSmZPHAsEdVN2daY31HDMhxVs70jz2yjvUVCU58ZAGTprRyNjaKsyUOESGKyvl7BgzOw3YCdzdRwKYABwCnAdsLSQAM0sCbwAfA9YCLwCXuPtrZnY/8JC7LzCzecDL7n77vuJobW31xYsX79cXHKj0rk6WrN3G1t2dTG2o581NO/lTejevb2jj9Q1tvLM9w+yp46hKGJnOHKcc2sQZMyfw1votvNsODy9et9eBO5kwjpoyloMb6+nK5ZneNAozY0d7FzMnjaG+uopX1m1ny65OqhKGGTz00joa6lN8bNZEzpo9mcb6ajbuaOed7e0AdOXyZDpzLFm3na27OjGDqpDIc00AAAuSSURBVESCs2dPYldnjlfX72DjjnZ2ZLpY+e4uqpMJdpZ4q4zxY2o4fMJo3j9xDEdNGcshTaOYMq6Wbbu7aBxVzZjaKrI5p74myYqNO1n57i5qqxJ0ZPP8Kb2bP/5pGzOa6/mz9zXzvvGjmXhQDTVVyQP/90inaWxsPOD1h0pc4gTFOlgqKVYze9HdT+xZXlILwN0Xmtn0fdRvAjaZ2Sd7VJ0MvOnuK8MgFgDnmtky4CPAX4bL/Ry4CdhnAiiHxlHVzDliQvd84SH0Bbm8k+yle2VmY4LGxka+/PEjWPbODv6wKs3OjiyZzhx/WJ3m2be2cFB9iqeWb6azj4vVJh9Uy66OLKe9fzy3XngME8ZEN2jd3pWjpirB2q0ZHvzDSqaOH8fpR4wnYcYLq9O8sbGNzpyzfluGFZt2cv+iNezu3P+7rB7SVM/CNzZzx+9WdZdNGlvL9OZ6pjXUM35M0I01fnQNTaOrGV1Txbj6FA311dSkkuzIdFGXSjK2Lmg17ezIkm1rJ9OZI9OVY3dnDnd48e00KzbupLoqwZRxdYyrT9E0qobDJowCIJVMMK4uSFh9dYdlOnN0ZvPU1yRJaexHRoDBHgNoAdYUza8FPkDQ7bPN3bNF5S29bcDM5gJzAVpaWkin4/HYx0wm0x3rxBr49JFju+uuOGl893Qu7+TdqUoY67Z3sGLzbg5prKWxPsW4uqKuoq7dpNO7I41xNzAKuHB2A3V1ddC5CwdOnFzNiZOb9lo2m3fWb2tn7fYO1mxtpyph7OzIkvegS6w9m6czm+dDhzZQXWV0ZJ3JY2uYclANma4cr23YxZqt7Wxs62D99g7eTmd4ankbW3d3kYvoEo3xo1N05Zxtmb5bNwaMqU0yYXQ1yYQxtjb4E9iyq4uVW/a01MaPTnV3gdVWJahKWPBKGqlkgpoqozqZoKYqEY7BBEmlI5vHgc6uLG4JunJONufUphLUphLk88G/ec6dVNKYOq6WmqoEOztyJAwOqq3CwzgTCSOZMFKJ4DOrkoVpoyoZxJRKhq9EWF80nc05uzpzuAc/UgqvwncpzHe2t5PLO2bB5/bW7VfoKSh3l2Dx31Wli0OsFT8I7O7zgfkQdAFVSpOqPwfS/GtqgmMOHaSA9qHUWCc0Q+sBfkbLxPG9lrs7uztzbG7rYMuuTnZ1ZNm6u5OtuzrZ1ZljbF2KrmyebZkukmaQ7aBx3BjqU0nqqpPUpZI4zozm0cxoDn7t7+7Msn5bO+ldnazflsEMsjln6+5OdmS62JbpYt3W4GC/dXcnCTMmN6Q4+5gWRtdWkekMxoZ2tmfJudORzZPN5cnmnI58nrawpdCRzdORzQXvXXkcpzaVDA6iQG0qSaoqQTJhtIctlmQi0X3wzXTlSO8akmGv/VZIBsmE4R78AOgpYZAwI5GwPdMWdFsG04TzwXQu73Tmgh8KPT8LwLAe84V66553dyxh3YmqOGkFDbviWAhjM/LuuAfrOwTTBGWF1l4u73t9z8K290wHc4X1CtspXrawnBnk83mSicRe36F424V9tb+ivKZ1sBPAOmBa0fzUsGwLMM7MqsJWQKFcRhgzY1RNFaNqqpgeHsD3pZRkVV9dxWETRkcV4gEpNam2tXeRzTmja6vIu7O5rYPqwgHJg9ZDNu90hQfOwnRXNk9nmJS6cntPB/NO0mB0bYpkAnJ5giSWd3LhNvIebHvnzt3U1tXtOUgSHig9iMEgHI+y4ABs1r1M0IINpvMeTBe2ky8qc3cSVmg9JfYcWX2vt+6Whr+nPHjPtGeoqantXrbnwTw4fnvQ0ip8ft67E1HhQF188O3M5TGCVlEiHHdz3/PpQeIoxOPdB3gLE053je+JBaC9o4Oa6pr3fAcn2MlOsI8OpE21vy2x3/VRPtgJ4AXg8PCMn3XAxcBfurub2VPABQRnAl0OPDLIsYhUnJ5nhBVfizJUKmmwsj+K9cD88OLey0s9DfQ+YA7QbGZrgRuBFIC7zzOzScAiYCyQN7PrgFnuvsPMrgaeIDgN9E53fzXc7NeABWb2beCPwM8O7KuJiMiBKPUsoEv6qd9A0I3TW91jwGO9lK8kOEtIRETKQOe6iYiMUEoAIiIjlBKAiMgIpQQgIjJCKQGIiIxQSgAiIiNUSXcDrRRm1ga8Xu44StQMvFvuIEqkWKMXlzhBsQ6WSor1EHd/z/1YKv5eQD283tstTSuRmS1SrNGLS6xxiRMU62CJQ6zqAhIRGaGUAERERqi4JYD55Q5gPyjWwRGXWOMSJyjWwVLxscZqEFhERKITtxaAiIhERAlARGSEikUCMLMzzex1M3vTzK6vgHimmdlTZvaamb1qZteG5Y1m9hszWxG+N4TlZmY/CuNfYmbHlyHmpJn90cweDednmNnzYUy/MLPqsLwmnH8zrJ8+xHGOM7MHzGy5mS0zsw9W6n41sy+F//5Lzew+M6utlP1qZnea2SYzW1pUtt/70cwuD5dfYWaXD2Gs/xz+H1hiZg+b2biiuq+Hsb5uZp8oKh/040RvsRbVfcXM3Myaw/my7teSBI9+q9wXwYNk3gIOBaqBlwkeNlPOmCYDx4fTY4A3gFnAd4Hrw/Lrge+E02cDjxM8P+4U4PkyxPxl4F7g0XD+fuDicHoe8Plw+gvAvHD6YuAXQxznz4GrwulqYFwl7legBVgF1BXtz7+ulP0KnAYcDywtKtuv/Qg0AivD94ZwumGIYv04UBVOf6co1lnhMaAGmBEeG5JDdZzoLdawfBrBg6/eBporYb+W9H3K8aH7ucM/CDxRNP914OvljqtHjI8AHyO4SnlyWDaZ4MI1gJ8AlxQt373cEMU3FXgS+AjwaPgf8t2iP7DufRz+J/5gOF0VLmdDFOdB4UHVepRX3H4lSABrwj/iqnC/fqKS9iswvcdBdb/2I3AJ8JOi8r2WG8xYe9SdD9wTTu/191/Yr0N5nOgtVuAB4FhgNXsSQNn3a3+vOHQBFf7QCtaGZRUhbMofBzwPTHT3d8KqDcDEcLrc3+GHwN8D+XC+Cdjm7tle4umONazfHi4/FGYAm4F/C7urfmpmo6jA/eru64BbgT8B7xDspxepzP1asL/7sdz/bwuuIPglDRUYq5mdC6xz95d7VFVcrD3FIQFULDMbDTwIXOfuO4rrPEjtZT/H1sw+BWxy9xfLHUsJqgia17e7+3HALoKuim4VtF8bgHMJktYUYBRwZlmD2g+Vsh/7Y2bfBLLAPeWOpTdmVg98A7ih3LEciDgkgHUE/WsFU8OysjKzFMHB/x53fygs3mhmk8P6ycCmsLyc3+FDwDlmthpYQNAN9C/AODMr3AuqOJ7uWMP6g4AtQxTrWmCtuz8fzj9AkBAqcb9+FFjl7pvdvQt4iGBfV+J+Ldjf/VjWvz0z+2vgU8ClYcJiHzGVK9b3EfwIeDn8G5sKvGRmkyow1veIQwJ4ATg8PLuimmAA7ZflDMjMDPgZsMzdv19U9UugMKJ/OcHYQKH8s+FZAacA24ua4oPK3b/u7lPdfTrBvvsfd78UeAq4oI9YC9/hgnD5Ifml6O4bgDVmdkRYdAbwGhW4Xwm6fk4xs/rw/0Mh1orbr0X2dz8+AXzczBrCFs/Hw7JBZ2ZnEnRbnuPuu3t8h4vDs6pmAIcDf6BMxwl3f8XdJ7j79PBvbC3BCSIbqMD92tsXqPgXwWj6GwSj/N+sgHhOJWg+LwEWh6+zCfp0nwRWAL8FGsPlDfhxGP8rwIllinsOe84COpTgD+dN4D+BmrC8Npx/M6w/dIhjbAUWhfv2vwjOkqjI/Qp8C1gOLAX+neDMlIrYr8B9BGMTXQQHpSsPZD8S9L+/Gb7+ZghjfZOgn7zw9zWvaPlvhrG+DpxVVD7ox4neYu1Rv5o9g8Bl3a+lvHQrCBGRESoOXUAiIjIIlABEREYoJQARkRFKCUBEZIRSAhARGaGUAERERiglABGREer/A4DN8+xmXz8NAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "not enough values to plot a chart\n",
      "Total time        : 0:26:07.003129\n"
     ]
    }
   ],
   "source": [
    "tfms = [None, TSClassification()]\n",
    "batch_tfms = TSStandardize(by_sample=True)\n",
    "dls = get_ts_dls(X_large_zarr, y_large_zarr, splits=splits, tfms=tfms, batch_tfms=batch_tfms, inplace=False, bs=[512, 1_024], num_workers=2) # 1*cpus\n",
    "learn = ts_learner(dls, InceptionTimePlus, metrics=accuracy, cbs=ShowGraph())\n",
    "timer.start()\n",
    "learn.fit_one_cycle(1, 1e-2)\n",
    "timer.stop()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>accuracy</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1.098602</td>\n",
       "      <td>1.098624</td>\n",
       "      <td>0.332375</td>\n",
       "      <td>24:27</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEXCAYAAACjyo8UAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de3xcdZ3/8ddnJpPJtU2TtE2bFnqFtlDaUtCKiBUEykVARaGLl8Xd7eJPt6CyiuIqrroPVNYfssvF4iKrclksKoooKMIWfxZoi6UEWiiFlqa3tJm2SdrcZub7++OcpGnJvWcyJ9P38/HII2fO98zM55xk3vOd7zlzjjnnEBGR3BXJdgEiIpJZCnoRkRynoBcRyXEKehGRHKegFxHJcQp6EZEcp6AXEclxCnrJeWa22czen+06RLJFQS8ikuMU9HJMMrO4md1qZtv9n1vNLO63VZrZo2a2z8wSZvaMmUX8ti+Z2TYzazSzV83sHH9+xMxuMLNNZlZvZg+ZWbnfVmBmP/Pn7zOzVWY2NntrL8caBb0cq24EFgBzgTnAO4Cv+m1fAGqB0cBY4CuAM7MTgc8CpzvnSoHzgc3+ff4JuAx4LzAe2Avc7rd9EhgJTAQqgGuA5sytmsjhFPRyrLoK+FfnXJ1zbjfwDeDjfls7MA443jnX7px7xnknhUoBcWCWmcWcc5udc5v8+1wD3Oicq3XOtQI3AZebWZ7/eBXANOdcyjm3xjnXMGRrKsc8Bb0cq8YDW7rc3uLPA/ge8DrwhJm9YWY3ADjnXgeuwwvxOjN70Mw67nM88Et/aGYfsB7vjWEs8FPgceBBf5jou2YWy+zqiRyioJdj1Xa8cO5wnD8P51yjc+4LzrkpwCXA5zvG4p1z9zvnzvTv64Dv+PffClzgnCvr8lPgnNvmfyr4hnNuFnAGcDHwiSFZSxEU9HLsiPk7RQvMrAB4APiqmY02s0rga8DPAMzsYjObZmYG7MfrmafN7EQzO9vfaduCN86e9h//LuDbZna8/xijzexSf/p9ZjbbzKJAA95QThqRIaKgl2PFY3jB3PFTAKwG1gEvAS8A3/KXnQ78EWgCVgJ3OOeewhufvxnYA+wExgBf9u/zA+DXeMM9jcCzwDv9tipgOV7Irwf+F284R2RImC48IiKS29SjFxHJcQp6EZEcp6AXEclxCnoRkRynoBfpg5lNMjPnf8tVZNhR0ItkmH8c/VNmtt/MNvewzLvM7C/+9DfN7CUzS5rZTUNZq+QmBb1I5h0A7gH+uZdlLsI71h+80y98EfhthuuSY4SCXoYdMxtvZg+b2W4ze9PMlnZpu8nMlpvZ//inEn7BzOZ0aZ9pZk/756R52cwu6dJWaGb/bmZb/N73n82ssMtTX2Vmb5nZHjO7sb/1Oueed879FHijl8UuxA9659x/O+d+BzT29zlEeqOgl2HFPy/8b4AXgWrgHOA6Mzu/y2KXAj8HyoH7gV+ZWcw/kdhvgCfwvtX6T8B9/umHAW4B5uOdj6Ycr1fd9VQFZwIn+s/5NTOb6dd0pn8is8Gu0zi8k5/9dbCPIdIbBb0MN6cDo51z/+qca3POvQHcDVzZZZk1zrnlzrl24Pt4pztY4P+UADf79/0T8Ciw2H8D+RRwrX8ispRz7i/+KYc7fMM51+ycexHvjWYOgHPuz865sqNYpwuB3zt9TV0yREcRyHBzPDD+iB50FHimy+2tHRPOubSZ1XLoFMRbnXNde+lb8D4ZVOK9IWyiZzu7TB/Ee9MIwoV4nzxEMkJBL8PNVuBN59z0XpaZ2DHh99Qn4J+CGJhoZpEuYX8c8BreicpagKl4vfUh4Q8nvRe4eqieU449GrqR4eZ5oNG/dmuhmUXN7GQzO73LMvPN7EP+ce/XAa14Z5N8Dq8n/kV/zH4h8AHgQT/47wG+7+/sjfqHPMaPtmD/erIFQMy7aQVmlu83nwms63rFKb+2ArzXZ56/fPRo65Bjl4JehhXnXArvwh1zgTfxeuI/wrsma4dHgCvwrtv6ceBD/sU/2vCC/QL/fncAn3DObfDvdz3eKYtXAQm8i4r0+Roxs/eYWVMvi5yFd2rkx/A+QTTj7RCGww+r7HC3v8xivGvbNnPoMociA6bTFEtO8b9gNM0597Fs19IfZvYKcLlz7pVs1yK5Sz16kSzxh29+opCXTNPOWJEs8YeSbs52HZL7NHQjIpLjNHQjIpLjQjl0U1FR4SZPnpztMvqUSqWIRofHUW+qNTNUa2ao1oFbs2bNHufc6O7aQhn0EydOZPXq1dkuo0+JRILy8vJsl9EvqjUzVGtmqNaBM7MtPbVp6EZEJMcp6EVEcpyCXkQkx4VyjF5EZCDa29upra2lpaVlyJ87nU6za9euIXu+goICJkyYQCwW6/d9FPQiMuzV1tZSWlrKpEmTMLMhfe5kMkle3tBEqXOO+vp6amtrGciRiRq6EZFhr6WlhYqKiiEP+aFmZlRUVAz4k4uCXkRyQq6HfIfBrGcog74tme57IRER6ZdQBn1rSuffEZHhY9++fdxxxx0Dvt+FF17Ivn2Dvq58v4Uy6FHOi8gw0lPQJ5PJXu/32GOPUVZ2NNeV759QHnXjlPQiMozccMMNbNq0iblz5xKLxSgoKGDUqFFs2LCB1157jcsuu4ytW7fS0tLCtddey5IlSwCYNGkSq1evpqmpiQsuuIAzzzyTv/zlL1RXV/PII49QWFgYSH19Br2Z3YN36bY659zJ3bTPAH4MnArc6Jy7pUvbZqARSAFJ59xpgVQtItKDb/zmZV7Z3tD3ggMwa/wIvv6Bk3psv/nmm6mpqWHt2rU8/fTTXHTRRdTU1HQeAnnPPfdQXl5Oc3Mzp59+Oh/+8IepqKg47DE2btzIAw88wN13381HP/pRHn74YT72sWAulNafoZt7gUW9tCeApcAtPbS/zzk3VyEvIseKd7zjHYcd537bbbcxZ84cFixYwNatW9m4cePb7jN58mTmzp0LwPz589m8eXNg9fTZo3fOrTCzSb201wF1ZnZRYFWJiAxSbz3voVJcXNw5/fTTT/PHP/6RlStXUlRUxMKFC7s9Dj4ej3dOR6NRmpubA6sn02P0DnjCzBzwQ+fcsp4WNLMlwBKAyvHHkUgkMlza0Wtubh4WdYJqzRTVmhkDrTWdTve54zNT0uk0hYWFNDY2kkwmSaVSOOc660kkEpSVlZGfn09NTQ3PPvssqVSqsz2ZTB423fGYva1TOp0e0PbJdNCf6ZzbZmZjgD+Y2Qbn3IruFvTfBJYBTJ5xsgvD+Z37EpbzUPeHas0M1ZoZA611165dQ3YagiMlk0nGjh3Lu9/9bubOnUthYSFjx47trOeiiy7i7rvvZvbs2Zx44oksWLCAaDTa2Z6Xl3fYNEAkEiESifS4TpFIZEDbJ6Nbxjm3zf9dZ2a/BN4BdBv0IiLD2f3339/t/Hg8zu9+97tu2zrG4SsrK6mpqemcf/311wdaW8aOozezYjMr7ZgGzgNqer+XT0dXiogEpj+HVz4ALAQqzawW+DoQA3DO3WVmVcBqYASQNrPrgFlAJfBL/7wMecD9zrnf96co5byISHD6c9TN4j7adwITumlqAOYMsi4REQlIOE+BICIigVHQi4jkuFAGvcboRUSCE8qgFxHJdSUlJQBs376dyy+/vNtlFi5cyOrVq4/6uRT0IiJZNH78eJYvX57R5whn0GvsRkSGmRtuuIHbb7+98/ZNN93Et771Lc455xxOPfVUZs+ezSOPPPK2+23evJmTT/ZODNzc3MyVV17JzJkz+eAHPxjY+W5CeT56Jb2IDNrvboCdLwX7mFWz4YKbe13kiiuu4LrrruMzn/kMAA899BCPP/44S5cuZcSIEezZs4cFCxZwySWX9Hjd1zvvvJOioiLWr1/PunXrOPXUUwMpP5RBr5gXkeFm3rx51NXVsX37dnbv3s2oUaOoqqric5/7HCtWrCASibBt2zZ27dpFVVVVt4+xYsUKli5dCsApp5zCKaecEkhtoQx6EZFB66PnnUkf+chHWL58OTt37uSKK67gvvvuY/fu3axZs4ZYLMakSZO6PUVxpoVzjF5EZBi64oorePDBB1m+fDkf+chH2L9/P2PGjCEWi/HUU0+xZcuWXu9/1llndZ4craamhnXr1gVSl3r0IiIBOemkk2hsbKS6uppx48Zx1VVX8YEPfIDZs2dz2mmnMWPGjF7v/+lPf5qrr76amTNnMnPmTObPnx9IXQp6EZEAvfTSoR3BlZWVrFy5stvlmpqaAO8C4R2nKC4sLOTBBx8MvKZQDt047Y0VEQlMKINeRESCo6AXkZzgjpGhgMGsp4JeRIa9goIC6uvrcz7snXPU19dTUFAwoPuFcmdsbv+pRCRoEyZMoLa2lt27dw/5c6fTaSKRoeszFxQUMGFCd9d66lkog15EZCBisRiTJ0/OynMnEgnKy8uz8tz9paEbEZEcp6AXEclxoQz6HN+fIiIypEIZ9CIiEpyQBr269CIiQQlp0IuISFAU9CIiOS6UQa+BGxGR4IQy6NtTinoRkaCEMugbWpLZLkFEJGf0GfRmdo+Z1ZlZTQ/tM8xspZm1mtn13bRHzeyvZvZof4vKj4by/UdEZFjqT6LeCyzqpT0BLAVu6aH9WmD9QIqKRmwgi4uISC/6DHrn3Aq8MO+pvc45twpoP7LNzCYAFwE/GlhZGqMXEQlKps9eeSvwRaC0rwXNbAmwBKCoagqJRI/vLaHR3Nw8LOoE1ZopqjUzVGuwMhb0ZnYxUOecW2NmC/ta3jm3DFgGMHLiiS7sp/2E4XF60g6qNTNUa2ao1mBlcq/nu4FLzGwz8CBwtpn9LIPPJyIi3chY0Dvnvuycm+CcmwRcCfzJOfexTD2fiIh0r8+hGzN7AFgIVJpZLfB1IAbgnLvLzKqA1cAIIG1m1wGznHMNgy1Ku2JFRILTZ9A75xb30b4T6PUChs65p4GnB1KYiIgEQ99MEhHJcaEMel1hSkQkOKEMehERCY6CXkQkx4Uy6J2OuxERCUwog15ERIITzqBXh15EJDDhDHoREQlMKINeHXoRkeCEMuhFRCQ4CnoRkRwXyqDX0I2ISHBCGfRKehGR4IQz6EVEJDChDHp16EVEghPKoBcRkeCENOjVpxcRCUoog14xLyISnFAGvZJeRCQ44Qx6EREJjIJeRCTHhTLoNXIjIhKcUAa9iIgEJ7RB75z69SIiQQht0KeV8yIigQhx0CvpRUSCoKAXEclxoQ165byISDD6DHozu8fM6syspof2GWa20sxazez6LvMLzOx5M3vRzF42s28MpDD16EVEgtGfHv29wKJe2hPAUuCWI+a3Amc75+YAc4FFZragv4VpZ6yISDD6DHrn3Aq8MO+pvc45twpoP2K+c841+Tdj/k+/41s9ehGRYORl8sHNLAqsAaYBtzvnnutl2SXAEoD8qmkkEntJFmS0vKPW3NxMItHje2CoqNbMUK2ZoVqDldEkdc6lgLlmVgb80sxOds51O9bvnFsGLAOIj5vuRo4sY1RxfibLO2qJRILy8vJsl9EvqjUzVGtmqNZgDclRN865fcBT9D7WfxgN3YiIBCNjQW9mo/2ePGZWCJwLbOjv/bUzVkQkGH0O3ZjZA8BCoNLMaoGv4+1YxTl3l5lVAauBEUDazK4DZgHjgP/2x+kjwEPOuUf7W5jOdSMiEow+g945t7iP9p3AhG6a1gHzBlmXevQiIgEJ7TdjNUYvIhKM0Aa9Yl5EJBihDfq0xm5ERAIR2qDXyI2ISDBCG/QaoxcRCYaCXkQkx4U46LNdgYhIbght0OsLUyIiwQht0KtHLyISjNAGfUpJLyISiNAGvXbGiogEI7RBn1SPXkQkEKEN+lQ6ne0SRERyQmiDPplSj15EJAihDXrtjBURCUZog15j9CIiwQht0KtHLyISjNAGvXr0IiLBCG3Q66gbEZFghDbo1aMXEQlGaINeY/QiIsEIbdDrOHoRkWCENujVoxcRCUZog75dO2NFRAIR2qBXj15EJBihDXqN0YuIBCO0Qa8evYhIMEIb9DqOXkQkGKENen0zVkQkGH0GvZndY2Z1ZlbTQ/sMM1tpZq1mdn2X+RPN7Ckze8XMXjazawdSmHr0IiLB6E+P/l5gUS/tCWApcMsR85PAF5xzs4AFwGfMbFZ/C9MYvYhIMPoMeufcCrww76m9zjm3Cmg/Yv4O59wL/nQjsB6o7k9Rhnr0IiJByRuKJzGzScA84LlellkCLAGIV02j6cBBEoke319Cobm5OfQ1dlCtmaFaM0O1BivjQW9mJcDDwHXOuYaelnPOLQOWARSOP8HF8gsoLy/PdHlHJZFIhL7GDqo1M1RrZqjWYGX0qBszi+GF/H3OuV/0+37oqBsRkaBkLOjNzID/AtY7574/sDtrjF5EJCh9Dt2Y2QPAQqDSzGqBrwMxAOfcXWZWBawGRgBpM7sOmAWcAnwceMnM1voP9xXn3GN9Pic6BYKISFD6DHrn3OI+2ncCE7pp+jNeZg+YGbSnNHQjIhKEUH4z1jBaFfQiIoEIZ9AbtCUV9CIiQQhn0KOhGxGRoIQz6M3UoxcRCUgogz6ioRsRkcCEMug1dCMiEpxwBr1Bq3r0IiKBCGfQY7SpRy8iEohwBr3G6EVEAhPaoNcYvYhIMEIa9Dq8UkQkKOEMejR0IyISlHAGvUG7zl4pIhKIcAa9f9SNcwp7EZGjFc6g909urEMsRUSOXiiDPuIHfUubgl5E5GiFNOi9pG9qS2a5EhGR4S+kQe/9bmpR0IuIHK1QBn3UT/qmVgW9iMjRCmXQdwzdHFDQi4gctZAGvfdbPXoRkaMX0qDX0I2ISFDCGfR+VdoZKyJy9MIZ9OrRi4gEJpRBb0A8L6KdsSIiAQhl0AOUFuTRqKAXETlqoQ36kniexuhFRAIQ3qAvyKOxpT3bZYiIDHt9Br2Z3WNmdWZW00P7DDNbaWatZnb9QO7bm9J4TDtjRUQC0J8e/b3Aol7aE8BS4JZB3LdHXo9eQS8icrT6DHrn3Aq8MO+pvc45twp42zhLX/ftTWlcQS8iEoS8bBfQwcyWAEsAqqurKYik2dPUwo66PcTzwrkrobm5mURiUO9jQ061ZoZqzQzVGqzQBL1zbhmwDGDu3Llu4uiRtCZ38J5bn+f1f7swy9V1L5FIUF5enu0y+kW1ZoZqzQzVGqxwdpWByZXFACTTjpb2VJarEREZvkIb9O+ZXtk5ve+gDrMUERms/hxe+QCwEjjRzGrN7O/M7Bozu8ZvrzKzWuDzwFf9ZUb0dN/+FlYcz+OcGWMA2HuwbcArJiIinj7H6J1zi/to3wlMGMx9+/IPZ03hyQ117D2goBcRGazQDt0AjCrKB2Cvhm5ERAYt1EFfXuwFff2B1ixXIiIyfIU66Cv8oF+24o0sVyIiMnyFOugj/sVja/c2Z7kSEZHhK9RBD/CPZ00BoLlNx9KLiAxG6IN+1vgRALyVOJjlSkREhqfQB33HN2Tf3HMgy5WIiAxPoQ/64yu8oN9cr6AXERmM0Af9yMIY5cX5bFHQi4gMSuiDHmBSRZGGbkREBml4BH1lMVvqtTNWRGQwhkfQVxSzY3+LDrEUERmE4RH0/pE3OsRSRGTghkXQTx9TAsD5t67g5e37s1yNiMjwMiyCfua4EZ3TF9325yxWIiIy/AyLoAf4zWfP7JzeuKsxi5WIiAwvwyboZ08Yyeqvvp9oxPjlX7dluxwRkWFj2AQ9QGVJnDOmVnDH05v4fc3ObJczKPVNrTS26EIqIjJ0hlXQA3z7stmMLo1zzc/W8J9/2pjtcg6z90Ab7ak06bTrtj2ddlx025+ZfdMTLLp1BS+8tXeIKxSRY9GwC/rjKor4xafPAOCWJ16jISS948aWds78zp+YfuPvmPKVx/i7e1dxoDV52DLXL3+RnQ0tAGzY2cin7l3FG7ubslGuiBxDhl3QA0wsL+KRz7wbgH/77fosV+P57bodHOjyha4nN9Sx6AcrqNm2n1/9dRtPvLyTX7zg7Vv44cfn8+0PnkxTS5JFP3iGNVsS3PbkRjbrNA8ikgF52S5gsOZMLONv3nkc9z/3Fn/zzuM4ZUJZ1mrZ3djK1379MqXxPD5z9jTOP6mKzXsOcPW9q7j4Pw4/HHTecWWcO3MskYgxo6qUD9+5kg/fuRKA7//hNR7+9BnMP35UNlZDRHLUsOzRd/jS+TMoiEX41qPrqW/K3gXEf7pyM23JNPd+6nSuee9UJlcW874ZY1h+zbs4Y2oFHzq1mkUnVbH07Gk89I/v6rxE4vzjy/nx1adzXHkRY0fEyY9G+PCdf+HzD63tcZw/lXa8uecAP1+9ldue3JjV9RaR4WHY9ugBRhbF+MAp4/n5mlo+9l/P89jSMzGzIa/jodW1AG/7VHHapHLu/4cFvd73fSeO4X1fHAPAG7ubuOQ//x+/eGEbjS1J/mPxPApi0c5lW5MpFi97lhfe2tc5b9XmBN+89OTO00SIiBxpWPfoAW665CTOmFrB+h0NvPd7T/PK9oYhff6abfvZ2dDCNe+dSix6dJtzyugSXrrpPD5/7gn84ZVdXPOzNYedyO0LD73YGfLvPWE0Z06r5JmNe1h4y9N89/cb2LZPF1EXkbcb1j16gOJ4HndeNZ8rlq1kw85GLrztGR74hwW8a2pFII//yNptXPvgWj73/hP40KnV3PW/m/jkGZM4YWwpX3rkNf7waj0Al84dH8jzmRlLz5nO6NI4X/7FS8z82u8ZUZBHQ8uhI3ie/MJ7mTraO//PH1/Zxd//ZDV3PL2JO57exLXnTOe690/PyicbEQknc677seBsmjt3rlu7du2A7/fTZ7fwL7+qAeC8WWP5l4tnMbG8aFA1bE0c5D3ffapfy14+fwK3fGTOoJ6nN13XB2DamBLuvfp0Jow6fJ3aU2m++egr/GTllsPmzzuujB9+fD5jSgtIJBKUl5f3+Fz3P/cWEYPzTqqivDg/2BUZoL5qDRPVmhmqdeDMbI1z7rRu23Ip6FNpxy1PvMpfNtXzUu0+0g4umzuev3/PFE6uHgl4X1ratLuJ1+uamHtcGbc/9TqjivIpLcijrCifuRPLmDa6hE/++Hme2bgHgNsWz6Nm235e29XIzv0t1O5tpsk/Rn7p2dP4zNnTiOdFe6zraLS0p/jBkxuJmvHZs6cdNmZ/pPZUmn9/4jXue24LjV0+AUwdXcwFMyv47Lmzur3/+255uvMKXiMLY3zzspO5ZE4wn1AGIywvnP5QrZmhWgfuqILezO4BLgbqnHMnd9M+A/gxcCpwo3Puli5ti4AfAFHgR865m/tT8GCDvqvf1+zgmp+90Hn7Q/OqmTK6mBUb9/D8m4l+Pca5s8byw4/N7zxKpoNzjvoDbazZuJ1z50x6W3sYpNOO+55/i1VvJthY18T6HQ1MGV3M//3oXOZMLGNr4iBLfrqG9TsO7dNYes50nly/i5e3N3D5/AncdMlJlMSHfnQvLC+c/lCtmaFaB+5og/4soAn4SQ9BPwY4HrgM2NsR9GYWBV4DzgVqgVXAYufcK30VHETQg3eUyu9rdnL9z1+kPXVoPT80r5ppY0toTzrmHVfG5MpiNuxsZM2WvazdupcRBTFKC2J8/ZJZjCiI9fj4YfkD98fyZzfytcc20ZpMc8LY0sMCflJFEXd/4jSmjy0lmUpz6x838p9PvU5+XoR3Ti7nS4tmdH4iGgrDabuq1sxQrQPXW9D32V1zzq0ws0m9tNcBdWZ20RFN7wBed8694RfxIHAp0GfQByWeF+XSudWcMbWS5WtqKS+OMWZEAQtPGP22nZUTy4s4d9bYoSptyJ19QgXnzpnE7U+9zgPPvwXA9eedwNXvnkxxl157XjTC9eefyJnTK/ntuh08snYbF//Hnxk3soAzplbSmkxRWRJndGmcwliUd0wup6GlnYmjijr3h+w90EbiYFvnDmMRya5Mfi6vBrZ2uV0LvDODz9ej0aVxPr1wajaeOlRGFsb4yoUz+cqFM0mnXa9DTgumVLBgSgX/vOhEfrpyC997/FUefsH7voAZ9PRBsLqssPMwz0vnjue8WVVccHJVKIe3RI4VoTm80syWAEsAqqurSST6N46eTc3NzcOiTji6WhfPKefyk99JU2uKkniUVNqxOdHCm/UH2dHQRmNLksTBdn5Ts7sz5I8bVcDjL+/kkbXb3/Z4UysLcQ6qywpYOG0UJ40rYWplEVH/zaBrra3JNPua2xlbGuel7Y1MrijMyn6Dnhwr/wNDTbUGK5OvmG3AxC63J/jzuuWcWwYsA2+MPgxjXn0Jy9hcfwRRa9eBrfFj4Ywj2m/c30IynaY0HmNkUYx02vGbddv5xm9eYUplMZvrD5BKOwryY7S0p3hm016e2eSdqrkwFmXqmGJK4nm0tbVTEM9nZ0MLb+w+/ERv8bwI844r49k3ElSXFVJenM/+5nZGFcWYf3w5VSPjtKccB9uSzKgawbzjyhhdGs/YUVHH2v/AUFGtwcpk0K8CppvZZLyAvxL4mww+n2RZ1ciCw25HIsalc6u5ZM74br/AlU47Ntcf4MXafbxU28Cm3U0kDrSxa38r+1oOMHV0CTOqSpldPZLnNyc47fhy8vMiPPdGPcX5Ubbta6ZqZAHHVxSxdus+Xqzt/sLxZjC5opjRpXHqD7RRXVbIydUjiEYiNDS3E4saze0pDKNqZAFF+VFq9zZT39TKgbYU5UX5jCsroGpEAQ0t7bSnHBXF+RTEosRpY3oyn3EjCw7b1yESJn3+Z5rZA8BCoNLMaoGvAzEA59xdZlYFrAZGAGkzuw6Y5ZxrMLPPAo/jHV55j3Pu5cyshoRZT9/SjUSMKaNLmDK6hA/OOzS/vr6ekpFlA+6F7zvYRlNrknhelFd3NpJ2ju37mtmxv4UNOxvYd7CdssIYm+sP8L+v7QZgREEeB9tSxKIR8vMi7G/2rm+QH41QWZLPiMIYa7fuY09Ta4/7JTqMLIwRi0ZoaU8RjRiVJfnEohH2HmyjLZmmzP++RiwawTnHiMIYVSMKmDamhHgsysZdjVQUx4nlGaXxPIrjeTQ0t3OwPUXUjIgZZhCNeNOptKO5PUVDSzttyTRF+VFK4jEi/lGXxkUAAAi8SURBVDKF+VGK873HyYsaTY2NlNWnOu/v/Xh/n4hBfl6EwliU8uJ8RhTEeGNPE3/aUMe4kYXk53nbJ+5vp86fLrfj0WjndDRitCZTHGxNkTjYRjwvQjwvSnsqTX5ehJJ4Hs7BgbYkB1tTHGhLcqA1SVsyTTRiNDQ2UuYfGNbx3xOJGFEzopFD28E4fB0iXbaT+bfNwDj0P9gxPy9i3mP6Q4b50QixqPX5rfJU2uGcI6+bU560p9Kk0s57bDPSzpF2kHaOZNqRdl5bx9/A6FKjGc5fvmOdgtKfo24W99G+E29Ypru2x4DHBleaHKvMbFBDLWVF+ZQVed/qHV0a73XZrYmDFMSijC6Nc7AtSTTiPWdzW4o9Ta1UjSw47NxFrckUe5raKIxFKcqPUn+gjZb2FJu27eagy2fH/ha27j3I828mKIkXMmv8CPYeaCOZdsyuHkk8FmHfwXYaW5Ik02kA6pvaeHHrPh5cdXQXz+kI2+b2FKkezno6UIWxKM3tqb4X7IYZxCIR2lLpQGoZSmZ4f3cHDm9b5vv/B2l/XlvSW69y/1NdOp2mJeloakke9TpHI3bY37DjTdv8N4WONy3vt/fmF4tGyOvjYAd91pRjUtdTYxTlH3oZFOZHuz1tRjwvSnVZYeftjulR0bajGp91zlG7txkzONCaYlJlEek0NLUmaWpNEosalSXxw3qG6bQ3fbAtScSM8X4tzjlak+nOnuPB1hQH27zHSaYdjQ0NFJWUkkx57c4PrrTzhtFak2kOtiXZsb+FuoYWxpcVctqkUYwoiNGaTNOWStOW7PLT5XZrl+mDbUnaU46i/CilBXmMLIzRlkzTnnbEIkZbKk1Tq1d7cX6Uwvw8ivOjFMXziOdFSKUdDY2NlJaUdoatc5Dqsu4dveqObeL89fe2z5HzDt/mHdswlT7Uy3bO6413rEtHmAK0JdOHQtaMeF4EA3Y3tdHanqK1rZXy0mKK4956RKNGMuU9ZsT/ZBGNRIhGvN570n/ujtpcxzr4t2PRyGF/l7RzpLw/Vuc6+Tf9dUiTTDlW9fJ/pqAXySIz6/aNpTA/2uenkiPPSWRmh53i4sgv+yUSLvQ7DTskEpFhVGs4dsZ+p5e2YX+aYhER6Z2CXkQkxynoRURynIJeRCTHKehFRHKcgl5EJMcp6EVEcpyCXkQkx4XymrFm1gi8mu06+qES2JPtIvpJtWaGas0M1TpwxzvnRnfXENZvxr7a0yWxwsTMVg+HOkG1ZopqzQzVGiwN3YiI5DgFvYhIjgtr0C/LdgH9NFzqBNWaKao1M1RrgEK5M1ZERIIT1h69iIgEREEvIpLjQhX0ZrbIzF41s9fN7IYQ1DPRzJ4ys1fM7GUzu9afX25mfzCzjf7vUf58M7Pb/PrXmdmpQ1xv1Mz+amaP+rcnm9lzfj3/Y2b5/vy4f/t1v33SENdZZmbLzWyDma03s3eFeJt+zv/b15jZA2ZWEJbtamb3mFmdmdV0mTfg7Whmn/SX32hmnxzCWr/n/w+sM7NfmllZl7Yv+7W+ambnd5mf8YzortYubV8wM2dmlf7trG7XfnP+Jayy/YN3AfFNwBQgH3gR7yLj2axpHHCqP10KvAbMAr4L3ODPvwH4jj99IfA7vMs5LgCeG+J6Pw/cDzzq334IuNKfvgv4tD/9f4C7/Okrgf8Z4jr/G/h7fzofKAvjNgWqgTeBwi7b82/Dsl2Bs4BTgZou8wa0HYFy4A3/9yh/etQQ1XoekOdPf6dLrbP8138cmOznQnSoMqK7Wv35E4HHgS1AZRi2a7/XKVtP3M3GfRfweJfbXwa+nO26jqjxEeBcvG/tjvPnjcP7ghfAD4HFXZbvXG4IapsAPAmcDTzq/+Pt6fJC6ty+/j/ru/zpPH85G6I6R/rhaUfMD+M2rQa2+i/WPH+7nh+m7QpMOiI8B7QdgcXAD7vMP2y5TNZ6RNsHgfv86cNe+x3bdSgzortageXAHGAzh4I+69u1Pz9hGrrpeFF1qPXnhYL/MXwe8Bww1jm3w2/aCYz1p7O5DrcCXwQ6LkNfAexzziW7qaWzTr99v7/8UJgM7AZ+7A8z/cjMignhNnXObQNuAd4CduBtpzWEc7t2GOh2DMvr7lN4PWMIYa1mdimwzTn34hFNoau1O2EK+tAysxLgYeA651xD1zbnvV1n9RhVM7sYqHPOrclmHf2Uh/ex+E7n3DzgAN4QQ6cwbFMAf3z7Urw3p/FAMbAoq0UNQFi2Y1/M7EYgCdyX7Vq6Y2ZFwFeAr2W7lsEKU9BvwxsD6zDBn5dVZhbDC/n7nHO/8GfvMrNxfvs4oM6fn611eDdwiZltBh7EG775AVBmZh3nM+paS2edfvtIoH4I6gSvZ1PrnHvOv70cL/jDtk0B3g+86Zzb7ZxrB36Bt63DuF07DHQ7ZvV1Z2Z/C1wMXOW/MdFLTdmqdSrem/2L/mtsAvCCmVWFsNZuhSnoVwHT/SMa8vF2Zv06mwWZmQH/Bax3zn2/S9OvgY696J/EG7vvmP8Jf0/8AmB/l4/RGeOc+7JzboJzbhLedvuTc+4q4Cng8h7q7Kj/cn/5Ien5Oed2AlvN7ER/1jnAK4Rsm/reAhaYWZH/v9BRa+i2axcD3Y6PA+eZ2Sj/E8x5/ryMM7NFeMONlzjnDh6xDlf6RzFNBqYDz5OljHDOveScG+Ocm+S/xmrxDtLYSQi3a7eytXOghx0gF+Id2bIJuDEE9ZyJ99F3HbDW/7kQb9z1SWAj8Eeg3F/egNv9+l8CTstCzQs5dNTNFLwXyOvAz4G4P7/Av/263z5liGucC6z2t+uv8I5KCOU2Bb4BbABqgJ/iHQkSiu0KPIC376AdL3z+bjDbEW98/HX/5+ohrPV1vHHsjtfWXV2Wv9Gv9VXggi7zM54R3dV6RPtmDu2Mzep27e+PToEgIpLjwjR0IyIiGaCgFxHJcQp6EZEcp6AXEclxCnoRkRynoBcRyXEKehGRHPf/AWKyRQgyI4rVAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "not enough values to plot a chart\n",
      "Total time        : 0:24:28.079497\n"
     ]
    }
   ],
   "source": [
    "tfms = [None, TSClassification()]\n",
    "batch_tfms = TSStandardize(by_sample=True)\n",
    "dls = get_ts_dls(X_large_zarr, y_large_zarr, splits=splits, tfms=tfms, batch_tfms=batch_tfms, inplace=False, bs=[512, 1_024], num_workers=4) # 2*cpus\n",
    "learn = ts_learner(dls, InceptionTimePlus, metrics=accuracy, cbs=ShowGraph())\n",
    "timer.start()\n",
    "learn.fit_one_cycle(1, 1e-2)\n",
    "timer.stop()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>accuracy</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1.098634</td>\n",
       "      <td>1.098615</td>\n",
       "      <td>0.333735</td>\n",
       "      <td>24:30</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEXCAYAAACkpJNEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXhU5dn48e89S/YEsrAGhLAIhFVAQUVFcUGqoK2+Qq3bT0u1tajVWlxabbWLr7bVWpXSira+KipqtYpLXRFF2WRTtsgiAbJAyALZJjPP749zkkwgK8xk5iT357pyOXOec07uOZJzz7Oc5xFjDEoppTofV6QDUEopFRmaAJRSqpPSBKCUUp2UJgCllOqkNAEopVQnpQlAKaU6KU0ASinVSWkCUJ2WiOwQkbMjHYdSkaIJQCmlOilNAEoFEZFYEXlYRPbYPw+LSKxdliEib4hIsYgUicgnIuKyy34hIrtFpExENovIFHu7S0Tmisg3IrJfRF4UkTS7LE5E/s/eXiwiK0SkR+Q+vepsNAEo1dBdwERgDDAaOAm42y67FcgFugE9gDsBIyJDgBuBE40xycB5wA77mJ8CFwFnAL2BA8BjdtlVQBegL5AOXA9UhO+jKdWQJgClGroc+I0xpsAYUwj8GrjCLvMBvYB+xhifMeYTY02m5QdigWwR8RpjdhhjvrGPuR64yxiTa4ypAu4FLhERj32+dGCQMcZvjFlljCltt0+qOj1NAEo11BvYGfR+p70N4EEgB3hXRLaJyFwAY0wOcDPWzb1ARBaKSO0x/YBX7SaeYmAjVsLoATwDvAMstJub/ldEvOH9eErV0wSgVEN7sG7atY6zt2GMKTPG3GqMGQBMB35W29ZvjHnOGDPJPtYAD9jH7wLON8Z0DfqJM8bstmsRvzbGZAOnABcAV7bLp1QKTQBKee3O2DgRiQOeB+4WkW4ikgH8Cvg/ABG5QEQGiYgAJVjf5AMiMkREzrI7iyux2vED9vnnAb8VkX72ObqJyAz79ZkiMlJE3EApVpNQAKXaiSYA1dktxrph1/7EASuBdcB6YDVwv73vYOA94CCwDHjcGPMhVvv/H4B9QB7QHbjDPuYR4HWsZqMy4HNggl3WE1iEdfPfCHyM1SykVLsQXRBGKaU6J60BKKVUJ6UJQCmlOilNAEop1UlpAlBKqU5KE4BSR0lE+ouIsZ/qVcpxNAEoFSH2cwAfikiJiOxoYp+TReQz+/V9IrJeRGpE5N72jFV1TJoAlIqcQ8AC4OfN7PMdrGcVwJqG4nbgzTDHpToJTQCqwxCR3iLysogUish2EZkTVHaviCwSkRfsKZtXi8jooPJhIvKRPWfPVyIyPagsXkT+KCI77W/rS0UkPuhXXy4i34rIPhG5q7XxGmOWG2OeAbY1s9s07ARgjPmnMeYtoKy1v0Op5mgCUB2CPS//f4C1QCYwBbhZRM4L2m0G8BKQBjwH/FtEvPYEbP8B3sV6ivenwLP2NM8ADwHjsObrScP6Fh48ZcMkYIj9O38lIsPsmCbZE8Ad7WfqhTVp3JdHew6lmqMJQHUUJwLdjDG/McZUG2O2AX8HZgbts8oYs8gY4wP+hDXtw0T7Jwn4g33sB8AbwCw7sfw/4CZ7Aje/MeYze2rnWr82xlQYY9ZiJaDRAMaYpcaYrsfwmaYBbxt9XF+FiY5eUB1FP6D3Yd+43cAnQe931b4wxgREJJf6qZ53GWOCv9XvxKpJZGAlim9oWl7Q63KsZBIK07BqKkqFhSYA1VHsArYbYwY3s0/f2hf2N/s+2FM9A31FxBWUBI4DtmBN8FYJDMT6dt8u7GapM4Br2ut3qs5Hm4BUR7EcKLPX5o0XEbeIjBCRE4P2GSci37XH7d8MVGHNzvkF1jf32+0+gcnAhcBCOyEsAP5kdzK77aGZsccasL1ecBzgtd5KnIjE2MWTgHXBK4TZscVh/d167P3dxxqH6rw0AagOwRjjx1pQZQywHeub+z+w1tyt9RpwGda6vFcA37UXZanGuuGfbx/3OHClMWaTfdxtWFNDrwCKsBZ7afFvR0ROE5GDzexyOtYU1IuxahwVWB3R0HD4Z62/2/vMwlq7uIL65SqVajOdDlp1CvaDU4OMMT+IdCytISJfA5cYY76OdCyq49IagFJRxm4G+pfe/FW4aSewUlHGbpL6Q6TjUB2fNgEppVQnpU1ASinVSTmqCSg9Pd1kZWVFOoxW8fv9uN3OGKHnlFidEidorOGisR6dVatW7TPGdDt8u6MSQN++fVm5cmWkw2iVoqIi0tLSIh1GqzglVqfECRpruGisR0dEdja2XZuAlFKqk9IEoJRSnZQmAKWU6qQc1QeglFJt5fP5yM3NpbKysl1/byAQID8/v11/Z1xcHH369MHr9bZqf00ASqkOLTc3l+TkZPr374+ItNvvrampweNpv1usMYb9+/eTm5tLa0dLahOQUqpDq6ysJD09vV1v/pEgIqSnp7eppqMJQCnV4XX0m3+ttn5ORyWACl+g5Z2UUkq1SmvmNF8gIgUisqGJ8qEiskxEqkTktsPKporIZhHJEZG5QduniMhqEVkjIktFZFBrgt1RVNGa3ZRSKmoUFxfz+OOPt/m4adOmUVxc3PKOx6A1NYCnganNlBcBc4CHgjfaKxU9hrXIRjbWAtvZdvETwOXGmDFYa57e3bawlVLKGZpKADU1Nc0et3jxYrp27RqusIBWJABjzBKsm3xT5QXGmBWA77Cik4AcY8w2e3rbhcCM2sOAFPt1F+rXZVVKqQ5l7ty5fPPNN4wZM4YTTzyR0047jenTp5OdbX0fvuiiixg3bhzDhw9n/vz5dcf179+fffv2sWPHDoYNG8YPf/hDhg8fzrnnnktFRWhaQ8I5RikTa6HuWrnABPv1dcBiEakASoGJTZ1ERGYDswFieg6iqKjJXBRVKioqNNYQc0qcoLGGy9HEGggE6r5t3//mRr7eWxbSmLJ7JXP3d4Y1+Xvvv/9+NmzYwMqVK/n444+ZPn06a9asISsri5qaGubPn09aWhoVFRWcfPLJzJgxg/T0dMCqJdTU1LB161aeeeYZnnjiCWbNmsWLL77I5Zdf3uTnbe01itRzALcA04wxX4jIz4E/YSWFIxhj5gPzAWJ7DTbRMrlSS6JpIqiWOCVWp8QJGmu4HE2s+fn5dePxxeUK+YggcbkaHe9f+xxAbZnH48HtdnPSSScxePDguv0ef/xxXn31VQB27drF9u3b6dGjR90xHo+HrKwsxo8fD8D48ePZtWtXk88YuFyuVl+jcCaA3UDfoPd9gN0i0g0YbYz5wt7+AvB2GONQSikA7rlweKRDIDExse71Rx99xHvvvceyZctISEhg8uTJjY7jj42NrXvtdrtD1gQUzmGgK4DBIpJlr3E6E3gdOAB0EZHj7f3OATaGMQ6llIqY5ORkysoab3YqKSkhNTWVhIQENm3axOeff96usbVYAxCR54HJQIaI5AL3AF4AY8w8EekJrMTq1A2IyM1AtjGmVERuBN4B3MACY8xX9jl/CLwsIgGshPD/Qv7JlFIqCqSnp3PqqacyYsQI4uPj65p3AKZOncq8efMYNmwYQ4YMYeLEJrtDw8JRawLH9hpsqvZujXQYrdLR21UjwSlxgsYaLkcT68aNGxk27MhO2nBr77mAajX2eUVklTFm/OH7OupJYKWUUqGjCUAppTopTQBKKdVJaQJQSqlOShOAUkp1UpoAlFKqk9IEoJRSUSQpKQmAPXv2cMkllzS6z+TJk1m5cuUx/y5NAEopFYV69+7NokWLwvo7NAEopVQYzZ07l8cee6zu/b333sv999/PlClTGDt2LCNHjuS111474rgdO3YwYsQIwJoFdebMmQwbNoyLL77YEdNBK6VUdHlrLuStD+05e46E8//QZPFll13GzTffzE9+8hMAXnzxRd555x3mzJlDSkoK+/btY+LEiUyfPr3JmUqfeOIJEhIS2LhxI+vWrWPs2LEhCV0TgFJKhdEJJ5xAQUEBe/bsobCwkNTUVHr27Mktt9zCkiVLcLlc7N69m/z8fHr27NnoOZYsWcKcOXMAGDVqFKNGjQpJbI5LAMaYkM/nrZTqJJr5ph5Ol156KYsWLSIvL4/LLruMZ599lsLCQlatWoXX66V///6NTgMdbo7rA/AHnDN5nVJKgdUMtHDhQhYtWsSll15KSUkJ3bt3x+v18uGHH7Jz585mjz/99NN57rnnANiwYQPr1q0LSVyOqwH4jXFe0EqpTm348OGUlZWRmZlJr169uPzyy7nwwgsZOXIk48ePZ+jQoc0ef8MNN3DNNdcwbNgwhg0bxrhx40ISl+PupQ6avVoppeqsX1/f+ZyRkcGyZcsa3e/gwYOAtSj8hg0bAIiPj2fhwoUhj0mbgJRSqpNyXgLQKoBSSoWE4xJAQGsASqk2ctLKh8eirZ/TeQmgc/x/VEqFSFxcHPv37+/wScAYw/79+4mLi2v1MY7rBNY+AKVUW/Tp04fc3FwKCwvb9fcGAgFcrvb9jh0XF0efPn1avb/jEkCgg2dxpVRoeb1esrKy2v33Hs0C9u3NgU1AmgCUUioUHJcAtAlIKaVCw3EJIBCIdARKKdUxOC4B6HMASikVGo5LANoHoJRSoeG8BKB9AEopFRKOSwDaBKSUUqHhvASgNQCllAoJxyUArQAopVRoOC4BaA1AKaVCw3kJQKsASikVEi0mABFZICIFIrKhifKhIrJMRKpE5LbDyqaKyGYRyRGRuUHbRUR+KyJbRGSjiMxpbcAdfUY/pZRqL62pATwNTG2mvAiYAzwUvFFE3MBjwPlANjBLRLLt4quBvsBQY8wwoNVrnfn1SWCllAqJFhOAMWYJ1k2+qfICY8wKwHdY0UlAjjFmmzGmGusmP8MuuwH4jTEmUHuO1gasfQBKKRUa4ZwOOhPYFfQ+F5hgvx4IXCYiFwOFwBxjzNbGTiIis4HZADE9B1FcWkpRkYQv6hCpqKigqKjJvBlVnBKrU+IEjTVcNNbQitR6ALFApTFmvIh8F1gAnNbYjsaY+cB8gNheg01SUlLUz7ENzpgLvJZTYnVKnKCxhovGGlrhHAW0G6udv1YfextYtYFX7NevAqNae1JtAlJKqdAIZwJYAQwWkSwRiQFmAq/bZf8GzrRfnwFsae1JdTI4pZQKjRabgETkeWAykCEiucA9gBfAGDNPRHoCK4EUICAiNwPZxphSEbkReAdwAwuMMV/Zp/0D8KyI3AIcBK5rbcA6CkgppUKjxQRgjJnVQnkeVvNOY2WLgcWNbC8GvtPKGBvQGoBSSoWG454E1umglVIqNByXAHQqCKWUCg3HJQCtACilVGg4LwFoBlBKqZBwXALQ5wCUUio0nJcAtA9AKaVCwnEJQKeDVkqp0HBcAtAHwZRSKjSclwC0BqCUUiHhuASgo4CUUio0nJcAtAaglFIh4bgEoMNAlVIqNByXALQGoJRSoeHABBDpCJRSqmNwXALQJiCllAoNxyUAHQWklFKh4bgEoM8BKKVUaDguAWgFQCmlQsN5CUAzgFJKhYSjEoCgTUBKKRUqjkoAoDUApZQKFUclABF9EEwppULFUQkARKeDVkqpEHFYAtAagFJKhYqjEoCgTwIrpVSoOCoBoH0ASikVMo5KAIImAKWUChVHJQDQJiCllAoVRyUAqwYQ6SiUUqpjcFQCQPRBMKWUChVnJQB0KgillAoVRyUAQbQPQCmlQqTFBCAiC0SkQEQ2NFE+VESWiUiViNx2WNlUEdksIjkiMreRY/8iIgfbErBWAJRSKjRaUwN4GpjaTHkRMAd4KHijiLiBx4DzgWxglohkB5WPB1LbEqyIjgJSSqlQaTEBGGOWYN3kmyovMMasAHyHFZ0E5BhjthljqoGFwAyoSw4PAre3NWDtA1BKqdAIZx9AJrAr6H2uvQ3gRuB1Y8zetpxQ0FFASikVKp72/oUi0hu4FJjcyv1nA7MBEnoOoKq6mqKiJiskUaOiosIRcYJzYnVKnKCxhovGGlrhTAC7gb5B7/vY204ABgE5IgKQICI5xphBjZ3EGDMfmA/Q9bghxuXxkpaWFsawQ6OoqMgRcYJzYnVKnKCxhovGGlrhTAArgMEikoV1458JfN8Y8xXQs3YnETnY1M2/MdoEpJRSodFiAhCR57GaazJEJBe4B/ACGGPmiUhPYCWQAgRE5GYg2xhTKiI3Au8AbmCBffM/BqKTwSmlVIi0mACMMbNaKM/Dat5prGwxsLiF45NaiqGWrgeglFKh46gngXU9AKWUCh1HJQCtASilVOg4KgGATgetlFKh4qgEoCuCKaVU6DgqAaBzASmlVMg4KgFoH4BSSoWOoxIAiE4HrZRSIeKoBCDobKBKKRUqjkoAuiawUkqFjqMSgNYAlFIqdByVABCo8WsCUEqpUHBUAhDA5w9EOgyllOoQnJUARKjRPgCllAoJZyUAwFejNQCllAoFZyUAAV9AE4BSSoWCsxIA4NNOYKWUCglnJQAR/AGjzwIopVQIOCoB1NJmIKWUOnaOSgAi1n/1WQCllDp2zkoA9n/f25gf0TiUUqojcFYCsKsANy1cE+FIlFLK+ZyVACIdgFJKdSDOSgBBGeCP726OXCBKKdUBOCsBBL1+9IOciMWhlFIdgbMSgGgjkFJKhYqzEkCkA1BKqQ7EWQlAM4BSSoWMoxLAsS4G5vMHuOOVdXy0uSA0ASmllIN5Ih1AWwSCMkD35NhG9/lwUwHdU2IZ3rtLg+2vfplLYoyH55fvYnNeGZOHdA9rrEopFe0clQBqO4FFoPBgFTX+AB63iwVLt/Pk0u28OWcS1zy9gpQ4D+vuPa/uuKJD1dzywtq69xU+nUtIKaUc1QSUHOvmL7NO4K5pwzAGyiprKK308Zs3vmZ3cQWb88oAKK2saXDcoaqG74sOVbVbzJGyPreEimp/pMNQSkUxRyUAgOmje5OaEANASYWPVTsP1JXtLq5o9JhD1Q0TQH5pFUu37gtfkBF24FA1F/51KT/818pIh6KUimKOSwAAXeK9gJUAqnz133K/LSo/Yt8vtu1n94EjE8MX2/eHL8AIy7U/79KcjpvklFLHrsUEICILRKRARDY0UT5URJaJSJWI3HZY2VQR2SwiOSIyN2j7s/b2Dfb5vW0JOiHGDUClz09V0BrBr6/dU/e6xh/A5w9w2fzPufaf9d+Ex/dLBaC6A68t/OWu+lpRR/6cSqlj05oawNPA1GbKi4A5wEPBG0XEDTwGnA9kA7NEJNsufhYYCowE4oHr2hJ0rNcKu7Im0CABbCs8VPd6bW4x+w9WNzjuhskDeX72RAD+tmQbBw/rG+go9pXV93Hc+er6CEailIpmLSYAY8wSrJt8U+UFxpgVgO+wopOAHGPMNmNMNbAQmGEfs9jYgOVAn7YEHeuxagBVPn/dN9zkWGtAU9cEqzKxOe8gBWWVDY6beWJfvO76j3z/G1+35dc6RkVQs9iiVbkUHapuZm+lVGcVzmGgmcCuoPe5wITgHeymnyuAm5o6iYjMBmYDZGZmUlRUROUhq61/f3EpxYesvBPvdVFWBX5/AI9L2LJnPwnSMCdVl5dRJPXfjt9Yu4fbz2xT7mm1iooKioqazJthVVzWsC9k7H3/ZeHVozi+e2Kj+0cy1rZwSpygsYaLxhpakX4O4HFgiTHmk6Z2MMbMB+YDjBkzxqSlpXGIOAC8cQm4fdYN/UCFdbMvq/LTPz2BwgrDnvKGjw5n9sggIcbD7NMHMH/JNmqMITU1NSyTzBUVFZGWlhby87aGce+ie3IsBUFNQav2VjFxaF8AyqtrOOdPS4jzuvjlBdn0TfSQGaFY2yKS17StNNbw0FhDK5yjgHYDfYPe97G3ASAi9wDdgJ+19cSxHrsPIKgJqHtyXF1537QEdhWV87ePtzU4Ls5uOrpz2jB+dUE2lb4Aq789gFO0timnwucnKa5hbi+pqK8NfVtUzu7iCr4pPMTVT61gyl9XNjmEVinVcYUzAawABotIlojEADOB1wFE5DrgPGCWMabNw1Tq+gBqApT7aojxuHjwklEALJ5zGselJbAut6TBTe+M47vhctV/0++XngDA955YdnSfrp39fck2xt73X97ekFe3rdLX+INeldV+4r1unr7mRJ69bgL90hPYU1yBP2AoqfBxcyNLav7rsx3hCl0pFaVaMwz0eWAZMEREckXkWhG5XkSut8t7ikgu1jf5u+19UowxNcCNwDvARuBFY8xX9mnnAT2AZSKyRkR+1Zaga0cBVdX4OVRVQ1Ksh1MGZbDjD98hu3cKx6UlHHHM4fPI9c+obw/3B45xlrkw27n/EL9dvBGgbiK719fuYegv3yanoOyI/Str/MR53Uwe0p1TB2XQq0scuw5UMPDOxYz+9btssp+YfurqE9n++2mANSpqj9YClOpUWjMKaJYxppcxxmuM6WOMedIYM88YM88uz7O3pxhjutqvS+2yxcaY440xA40xvw06p8feNsb++U1bgq5tAqryBThU5Scx1t2gvFfX+MY+R4P3A7slcU52D+v1nYspKT98EFP02Lavfnhrfqk1sumjTVYiWL2zuMG+b67by6c5+/EE1XZ6d4ln7a6G+wGccFxXRITLx/cC4I11e47YRynVcTnySWARIcbjoqomwMGqGhJjGrZ3J8e1rm/7e2Mz616P/s27IY0xlCqD5vT5em8pACn209D7guY1+mBTPj95bjUAX2yvH33Qq2t9/0it+y4aQVd7So1bz+pPRlIsv1u8iakPLwn9B1BKRSVHJgCwagG1TwLHeRvWABKC3g/tmUxGUgxzpgw+4hzdmphSOtpU1lgJILNrPPmlVfSf+yZvbdgLwEebCqn0+Smt9PFpTuPTW/RMOTIBJMY0vGbH90gCYFNeGfsPdvzJ8pRSkR8GetTivG6qagLU+AN43Q2HcQY36f/r2pMajBAKVjunULSrtKevvn3qEG6yO3DzS62b9PIdRQz95dvNHh/88FutGE/DbWcc343PvrESyJb8g5yc5IzkqJQ6eo6uAVT5/NT4DR5Xw49xYv/UutdN3fzBGi46rFcKJw9IB1o/zDKUKn1+3vs6n0AzHdF//SAHgAlZ6a065/TRvXno0tFHbD9tcAa3Tx0CwLh+qQ3KrjttQN3rrY10LCulOh7HJoD0xBjyyyrxBQJ4DqsBeBr5xtuYWI+bt246jZvOtpqHnlm2M+RxtmToL9/mun+tZOojSygPmrb6929t5NqnVwD101x3TfDy/QnH1e3zw9OyePeW0484519mncAl4+qfcL5gdG/Oze7BA98bxY8nD2Lb76bRq0vDjnK3Szh7mLVK2pZ8TQBKdQaOTQBZGYks317EwcqaBiNejsYJx3UFIvvNd0v+Qe58xZq4zecP8LePt/H+pgLKKq3RSR6XEOd1c8XEfnXHXDGxP8f3SOaj2yY3e+6kWA/zrxxPb3t0lKuJ6/WPq05kXL9UtuQdBKxhtv3nvsl5f16iD4op1QE5NgGM65eKz28oKKtq9Bv/l788hxV3nd2qc8V63Jw6KL3db3KHD0399xprGGbwN/DPt1mjee650JpIdVivFJ68ajw3njmIvmnWDb1/RiKj+lhrIH8296xjiql7cizLdxSxZEsh+SVWP8Pm/DJO/cMHrNwR3fOaKKXaxrEJIC3R6qQsqfA1WgNITYxp0yifzK7xjS4cc7QqfH4ufvzTZqeaqPYf+RB0aaWPXUX1cdSu6pUR1Ck7ZVgPbjtvSIM5jBZdfwqb7pta9y3/aNVOqX3lguWsyW347MC7X+cf07mVUtHFsQkg+OGv1rb5NyezawIFZVVU1YRmHd1v9pXz5bfFfPfxz8grqWx0n8rqIxPAqh0HeOKjnCO2j+7btdnfF+NxHTEc9mhcPrG+j2HO818CcPUp/QGYv2QbL6z4ttnjN+wu4ZqnlvPK6twGfRpKqejj2AQQ/LCX9xj7AKD+YanaETetVV0T4NYX15JTcLDB9vKgm/tTn21v9NiF9s00eEz+NU+vYG1uCQC1H+t/xvc55m/2rXXlyf2P2Hbrucfz8/Os0UO/eHl9k3MQATz92Q4+3FzIz15cy12vNrqInFIqSjg2AQzukVz32h2CBNDPnj/o0Q9y2Li3FGMMlzzxGa+szm32uOXbi3h5dS5n/+njBtsPBE0tsTX/4OGHAfD7tzYB8IfvjeKT289s8Dle+8mpdZPepdpP7LaXl284ucH75DgvM8b0rnu/fHvTfQHBTVUfbynkZy+sqZu+QikVXRybAFLivJwy0BoXv87+xnwsTspKo3cXqxZw/iOfsOyb/azceYCfvbi2yWPySyv5wZNf1L2vbe/PK6nkjv9sBWBIj2Q+2FTAlQuWNzi2Imh6h9SEGPqmJRCcxob1SuFse66ic4f3OKbP1lbj+tXPYf76jacC0Cc1gcvtIaj/Wdv0nEHBtYOiQ9W88uVu/vju5jBFqpQ6Fo5NAACTBmcAkFPY+DfsthAR/veS+oenvv+PL5rZ21I7L0+t7z7+GetzS1i100oELrFG0AAs2VJI7oH6lbpqk0VyrIfx9oNrtX26n809ixiPi19PH85/bzm9wQ25vTx4ySgeunQ0o/rU9z389uKRXDquD29vyGtyBtXDm8IATHRPtqpUp+XoBHD96QN5ZOYYlt85JSTnmzQ4g6euPvGI7Y3d7Cp9fn70r1WA9eRtrQv/urRuQrbP5k7hF1OH1pUFj+7595e76RLvZcXdZ9d13va1m6Fq1zVOS4xp0NTVni4d37fBw2S1JgxIp6yqhtsXrcMYw9SHl/Dzl6xa0r6DVSzN2QfAo7NOqDvGrxlAqajk6ATgcgkzxmSSHsJ5a84c2p0Xf9SwDXxz3pEPiG0rPES1P8CkQRn8ZdYJbPvdtCP26ZYcy/VnDODhy8YAsMuuAewpruClVbm4hAYjd569bgIPXzaGhJjonaJpZKb1vMHLq3O58K9L2ZRXxkurcrnxudWsCxo2emFQUtxVVH7EeZRSkefoBBAuJ2WlNXiGoHbmzWCrdlodofddNAKwktGvpw9vsI/bJYgIF47uTWqCl1U7DlBeXcPfPv4GaNhRDNCrSzwXnZBJNBvYrX4hnQ2765E9HV0AABVgSURBVJvA3li3ly+2Newcru043rC79IiH3pRSkacJoAlv/HQS916YTYzbxaMf5PDuV3kNyvNKK3G7hP7p9auPXXVKfz65/UwAxh+XUrfd7RL6piWwt7SSHz2zin/acw7NPX8oTtPcMxd/W2KtwVx7DR6ZeQL3zRhOhc9PfmkV736Vh6+Rh9+UUpGhCaAJPVLiuPrUrLqndWc/s6pBeaUvQJzH1eBpXLDa8TfdN5XH/ye7wfbjeySzckcRn2zdV7ft+jMGhin68Bp7XMOH0v7XXo+5Vo+g9Qdql978+yfbmP3MKn5nL22plIo8TQBtUNuMEQgYist9TT55G+d1HzE9xQ8m9qO8OjRPGUfayzecwklZ9SOTDl+DOXitgUHdrYVmnlxqPQz30srmn6tQSrUfTQAteGTmmLrXWXcs5o11e3jw3c28vDqX/W1YP2BIhEbzhIOI8NTVJ/L3K8fz5FXjmTggnWeuPanRfQ+fdvpgVQ39575J/7lvhmzaDaXU0dEE0IIZYzJ5/9Yz6t7f+NyXvL+x7ZOixce4eeXHpzDB/ubc2nWLo1VirIdzsnswZZj1kNppg7uRENN4jSg1ofGV1w5f0F4p1b6cfRdqJwO7JTEhK61uofXaKRraauxxqbzwo5P5fNv+Ixay7wiW3TGl0U7eN+ecRnm1n0qfnwseXVq3/bEPczh5YOtWOVNKhZ7WAFrpge+Nqpt/f/3uEsb1S21xIZamTByQzkh7/v6OpEu8t8FcQLV6d41nUPckju+RTHavFG48cxApcR6W5uzj31/u1iGiSkWIJoBW6p+RyD+urH9KuG9qfN0IF9U6MR4Xb86ZxG3nDeG047sBcPMLa8i6Y3GzM4wqpcJDE0Ab9Emt79A8ZWBGBCNxrtphs3d/ZxgZSfWznH60uSBSISnVaWkCaIPEWA/r7z2Xpb84k/85sW+kw3G0Xl3iWXn3OXXvH/0gp8EMqUqp8NME0EbJcV76pCa0vKNqlf+7dgLXTcriqz2l3PrSmkiHo1SnoglARdSkwRncfUE2sR4Xi9fn8aE2BSnVbjQBqKhQVWMNH12ypTDCkSjVeWgCUFGhdh2Gpz7doU8IK9VONAGoqHDm0O7E2nMIhWKJT6VUyzQBqKjx31usKTdW7jjQ5JKTSqnQaTEBiMgCESkQkQ1NlA8VkWUiUiUitx1WNlVENotIjojMDdqeJSJf2NtfEJGYI8+sOpvj7LUVHnh7Ez86bPptpVTotaYG8DQwtZnyImAO8FDwRhFxA48B5wPZwCwRqZ0k/wHgz8aYQcAB4Nq2ha06qism9gPgvY35PPjOJq0JKBVGLSYAY8wSrJt8U+UFxpgVgO+wopOAHGPMNmNMNbAQmCHWo6BnAYvs/f4JXHQ0wauO576LRvDLC6zvCY99+E3dIvNKqdAL55SUmcCuoPe5wAQgHSg2xtQEbW9yIVwRmQ3MBsjMzKSoqMlcFFUqKio01qN0cXYXuscN5aeLNnH9Myt55oqRDMhIoLy8nFXflvDgBzt46vsjiG9i+uloEG3XtDkaa3g4Idaon5PYGDMfmA8wZswYk5aW1sIR0aGoqAiN9ehdmJZGJTH8fNE6nlyezxM/GMdd/9nKWxutGkFelYdxPVMjHGXTovGaNkVjDQ8nxBrOBLAbCJ4wp4+9bT/QVUQ8di2gdrtSDVw6vi9f7irmuS++Zegv36LSV7/WwNOf7WBcv+hNAEo5QTiHga4ABtsjfmKAmcDrxpr8/UPgEnu/q4DXwhiHcrDzhvcEaHDzB/jP2j0sXP6triWg1DFosQYgIs8Dk4EMEckF7gG8AMaYeSLSE1gJpAABEbkZyDbGlIrIjcA7gBtYYIz5yj7tL4CFInI/8CXwZGg/luooThvUcNrtJT8/kxiPi4m/f5+5r6wnJd7LtJG9IhSdUs7WYgIwxsxqoTwPqxmnsbLFwOJGtm/DGiWkVLNcLuHtm08jMcZDbKCC7vazAtdNyuIfS7ezeucBpo3sxaGqGg5V15CaEIPXrc83KtUaUd8JrNTQnikAFBVV1m27+4Js3tqQxz+WbsfjdrFyRxErdx4g3uvm5rMH86MzBkYqXKUcQ78qKcfaXVwBwLyPv2HlzgMAVPj8/P6tTWzYrfMJKdUSTQDKsX48uelv+Rc8upSH39vSjtEo5TyaAJRjzT59AEN7Jte9H3tcV9746SSuPqU/AA+/t5Xqmoajh7bvO8R/1u7R0UNKoX0AysG6JsTw9s2nA7ByRxFDeiaTHOdlRGYXzji+G9c8vYLj736LQd2TyCk4yFUn92Pj3jKW7yjip89/ydWn9OeHpw/gv1/lMWlwNwZ1T4rwJ1KqfWkCUB3C+P4Nn7g8/fhuXDGxH898vpOcgoMA/HPZzgb7PP3ZDj7YVMC3ReUAvDB7IhMGpLdPwEpFAW0CUh2S2yXcd9EIZp8+AICRmV0a3a/25g9w60tr8fkDje6nVEekCUB1aDefPZjrzxjIU9ecyLSRPRGBJ68azyXj+vDjyQPpn57A8jun8I8rx5N7oII//3cLAZ2CWnUS2gSkOrSEGA9zzx8KwGPfH0t5tZ/EWA9ThvUA4OfnDUFEOGtoLOdk9+Dxj74hp+Ag834wDpdLIhm6UmGnNQDVaYgIibGeI7aB9cTxo7NO4JSB6bz7dT4/fnZ1JEJUql1pAlDKFud18+x1E/je2D68/VUe/ee+SXF5daTDUipsNAEoFURE+O3FI+reP798F19s288pv3+f19bsbvT5gbW7ivlka2F7hqlUSGgCUOowcV43b/x0EmAtUH/Z/M/ZU1LJTQvXkHXHYlbttFZ5Kiyr4tv95cx+ZiVXPLmcj7doElDOop3ASjViRGYXxvdLrZtjCOD6MwYy7+Nv+N4Tyxo95qoFyzl/RE+uP2MgfRObP3+lz8//fb6TmScdR1Ks/hmqyNAagFJN+NWF2fRLT+Cc7B48fNkY5p4/lOsmZTW67yMzxzCuXypvbchjxmOf8vQXuymr9AGwcPm33P/G15RV+qjxB7jntQ0M/eXb3P/mRiY/+CFrdxXz7f7yRs+rVDiJk+ZEGTNmjFmzZk2kw2gVJ6wHWsspsUZLnOXVNXy8uZDyaj+3vrQWgG2/m0bAGEbc+06D1cteuv5kLp1n1RgmZKVxyznHM3P+53XlyXEeyiprAPjNjOF8d2yfuhpBIGB45P2tfLWnlBsmD2Bcv7S67aEcohot17U1NNajIyKrjDHjj9iuCSA8oul/fkucEms0xvnHdzcT53XzkzMH1W37cFMBd7yylrzS+hFEGUkx7DtY//7sYd25+zvZ7Cmp4C/vb+XzbVa/QozHxbWTsrhiYj8+2FTA3f/eUHfMgG6JDO2ZzOL1eQD8z/g+3HjmYI6zF8kBMMbUDW1trWi8rk3RWI+OJoB2Fk3/81vilFidEifAvv37efbLIv5sT0n9wa1ncNYfPwYgLTGG1b88p8H+Pn+Atzfk8crqXD7eUoiI4LefSP73T07losc+PeJ3eN2Cz2/tc252D84f2ZPfvrmJPqnxnHF8N8b2S2VkZhdyD5Qz/a+fMjKzCyMyu3DJuEzGHpdad+zB0mLHXFcn/RuIplibSgDa+6RUGLhEuOnswYzITOHJpds5Li2BV358Cn/+7xbunDbsiP29bhcXju7NhaN7s33fIc586CMAJg5IY0zfrsw+fQDzl2wDYM5Zg/jZuUPIPVDOpAc+BODdr/N59+t8APYdrGLNruK6cw/sZvVIr99dwvrdJTy//NsGv3tadgbfHd+fUwdlEB/jPurPXF0ToLTSR0ZS7FGfQ7UvrQGESTRl/5Y4JVanxAnHHmtOQRn3vP4Vj84aS1piDD5/gN0HKiiu8DEyswtuuw9g495SCsqqeOS9Lfj8hgVXn0hqgpdH3t/Kox/k1J3vknF9eOjS0ZRX1/DMsp088PYmAsaqjVT7/Bys9uN1CxMHpCMi9E9P4MazBuHzG2b/ayWTBmdw05TBvLwql+N7JOMPGFLivQztmczzy79l+75y3t6wlz0llfTuEke35FhiPW76pSdw8QmZnDzQmmW1sKyKbsmxbW6mCtV1bU/RFKs2AbWzaPqf3xKnxOqUOCE6Yq30+Xll9W4WfLqdP146mtF9u9aVVVT7ySutJCsjkfzCfWw+YPhocyGvfJlLcbk1einO62JIj2TW5rZuec3kOA/TRvSi2h9g+75DbM4rw2DqOsV7psSRV2qt65yRFMP3J/Rj5Y4iDpT7mDq8Jycc15WB3ZPYue8QPbrEkRjjoUdKw2QRDde1taIpVk0A7Sya/ue3xCmxOiVOcG6sPn+AV1bn8vLq3SzfbnVM3zltKHuKK/lyVzETs9Lol55In9R4duw/xAebChiV2YUhPVM4eWA6aYkxdec1xlBVE+D3izfyz2U7ife6GZGZwo795RSWVbUpxsQYN9ecmsWkfglMHNq3yf0CAcOCT7fz/sYC4mPcHCiv5sChagZ2S+K0wRlsLThIfmkVMR4h3uvB7QKf35AU68HnD9A1IYaUeA+pCTFUVPvpEu9leGYKPVPi2LbvEAWllaTEe/G4XHy9p4SNe8vo0SWObkkxlFT4OFTtJynWQ5zXjb+6kvQuycTHuHEJlFbWYIwhLTGWxBg3XRK8dE2IocZOmMlxXrrEWz8p8R5i3K5W15TKq2vIKThIXkklB8qrKanwER/jISXOw57iSn585iBNAO3JqTeAaOaUOKFjxPryqlxcLrhoTOZRN9mAlQgKD1bRPTmublt+aSW7isoZ3COZwrIqjDFs23eI7fsOUV5Vw4FyHy6B51fsOmJZz6RYDwerahjQLZHC0iqG9U4hJc66aX+wqYD9h4JHX8UyISuNNbuK2V1cQXKsh8zUeGoChopqPwFjMMa6gcZ63Rw4VE1NG6YDT0uM4UB5NbW3UY9L2nR8c0Qgzm5GS4hxs7ekkliPi1iPG49bKDpUjdslBAKGPSWVzZ5r5wMXaAJoTx3hBhBtnBInaKyhUjus9Y11e1i18wAfb8pn2/6KZo+5aExvfjplMMlxHlLivMR53QQChtwDFWSmxtf1nzSlvLqG/QerifW4KKnw8fXeUuu918XQnink2TfbIT2TGNQ9meqaAMXl1aTEe4n1uKiqCVDlC5BXuJ+4pGQqfH4CAWuIr0ugyE4yxeU+Sit8BIyhu93UVVrho6TC2l5aWUN1TYCNe0up9gcYkJGEzx+guiZAtT9A1wSvfZGgb1oCA7olMrBbEqmJMcS4XVTV+Kn0+UmI8ZCZmqCjgJRSzlJb87hgVG8uGNWbolN7kZTSlWp/AH/A8Pzyb9maf5BuybGUVFSz72A1D146Gq+74SQHLpc0eF6iOQkxHhLSrFtj95Q4BvdIbnb/GI+L7in1tZs4r9tqAkqOIS3tyDlBBnRrVRjtQhOAUspRYjwuYjzWDf76MwZGOBpn07mAlFKqk9IEoJRSnZQmAKWU6qQ0ASilVCelCUAppTopTQBKKdVJaQJQSqlOShOAUkp1Uo6aCkJEyoDNkY6jlTKAfZEOopWcEqtT4gSNNVw01qPTzxhzxDPITnsSeHNj81lEIxFZqbGGllPiBI01XDTW0NImIKWU6qQ0ASilVCfltAQwP9IBtIHGGnpOiRM01nDRWEPIUZ3ASimlQsdpNQCllFIhoglAKaU6KUckABGZKiKbRSRHROZGQTx9ReRDEflaRL4SkZvs7Wki8l8R2Wr/N9XeLiLyFzv+dSIyNgIxu0XkSxF5w36fJSJf2DG9ICIx9vZY+32OXd6/nePsKiKLRGSTiGwUkZOj9bqKyC32//8NIvK8iMRFy3UVkQUiUiAiG4K2tfk6ishV9v5bReSqdoz1QfvfwDoReVVEugaV3WHHullEzgvaHtb7RGNxBpXdKiJGRDLs9xG9pq1mjInqH8ANfAMMAGKAtUB2hGPqBYy1XycDW4Bs4H+Bufb2ucAD9utpwFuAABOBLyIQ88+A54A37PcvAjPt1/OAG+zXPwbm2a9nAi+0c5z/BK6zX8cAXaPxugKZwHYgPuh6Xh0t1xU4HRgLbAja1qbrCKQB2+z/ptqvU9sp1nMBj/36gaBYs+17QCyQZd8b3O1xn2gsTnt7X+AdYCeQEQ3XtNWfKVK/uA0X/WTgnaD3dwB3RDquw2J8DTgH6ynlXva2XlgPrgH8DZgVtH/dfu0UXx/gfeAs4A37H+W+oD+wumts/0M+2X7tsfeTdoqzi31TlcO2R911xUoAu+w/ZI99Xc+LpusK9D/sptqm6wjMAv4WtL3BfuGM9bCyi4Fn7dcN/v5rr2t73ScaixNYBIwGdlCfACJ+TVvz44QmoNo/tFq59raoYFflTwC+AHoYY/baRXlAD/t1pD/Dw8DtQMB+nw4UG2NqGomnLla7vMTevz1kAYXAU3Zz1T9EJJEovK7GmN3AQ8C3wF6s67SK6Lyutdp6HSP977bW/8P6Ng1RFquIzAB2G2PWHlYUVXE2xQkJIGqJSBLwMnCzMaY0uMxY6T3iY2xF5AKgwBizKtKxtIIHq4r9hDHmBOAQVlNFnSi6rqnADKyk1RtIBKZGNKg2iJbr2BIRuQuoAZ6NdCyHE5EE4E7gV5GO5Wg5IQHsxmpjq9XH3hZRIuLFuvk/a4x5xd6cLyK97PJeQIG9PZKf4VRguojsABZiNQM9AnQVkdq5oILjqYvVLu8C7G+nWHOBXGPMF/b7RVgJIRqv69nAdmNMoTHGB7yCda2j8brWaut1jOjfnohcDVwAXG4nLJqJKRKxDsT6ArDW/vvqA6wWkZ5RFmeTnJAAVgCD7dEVMVgdaK9HMiAREeBJYKMx5k9BRa8Dtb36V2H1DdRuv9IeGTARKAmqioeVMeYOY0wfY0x/rGv3gTHmcuBD4JImYq39DJfY+7fLN0VjTB6wS0SG2JumAF8ThdcVq+lnoogk2P8eamONuusapK3X8R3gXBFJtWs859rbwk5EpmI1W043xpQf9hlm2qOqsoDBwHIicJ8wxqw3xnQ3xvS3/75ysQaH5BGF17RRkep8aGPHyzSskTbfAHdFQTyTsKrP64A19s80rDbd94GtwHtAmr2/AI/Z8a8Hxkco7snUjwIagPWHkwO8BMTa2+Ps9zl2+YB2jnEMsNK+tv/GGikRldcV+DWwCdgAPIM1MiUqrivwPFbfhA/rxnTt0VxHrPb3HPvnmnaMNQerrbz272te0P532bFuBs4P2h7W+0RjcR5WvoP6TuCIXtPW/uhUEEop1Uk5oQlIKaVUGGgCUEqpTkoTgFJKdVKaAJRSqpPSBKCUUp2UJgCllOqkNAEopVQn9f8BYefEJPlnFCYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "not enough values to plot a chart\n",
      "Total time        : 0:24:31.011945\n"
     ]
    }
   ],
   "source": [
    "tfms = [None, TSClassification()]\n",
    "batch_tfms = TSStandardize(by_sample=True)\n",
    "dls = get_ts_dls(X_large_zarr, y_large_zarr, splits=splits, tfms=tfms, batch_tfms=batch_tfms, inplace=False, bs=[512, 1_024], num_workers=8) # 4*cpus\n",
    "learn = ts_learner(dls, InceptionTimePlus, metrics=accuracy, cbs=ShowGraph())\n",
    "timer.start()\n",
    "learn.fit_one_cycle(1, 1e-2)\n",
    "timer.stop()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's great. An interesting finding is that most of the time the GPU utilization is 100%. This means the batch creation process is no longer a bottleneck. That's why speed is much faster. To further reduce time, we'd need a better GPU and/ or multiple GPUs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Setting num_workers>0 helps a lot because the bottleneck is batch creation, not the GPU. \n",
    "\n",
    "In any case, the goal is to ensure maximum utilization of your GPU. You can easily get a view of GPU utilization by running this code in your terminal during training: \n",
    "\n",
    "```\n",
    "watch -n .2 nvidia-smi\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I've run some additional tests (look below) to measure the impact of number of workers on a dataloader that uses memmap arrays vs zarr arrays. Here are the findings: \n",
    "\n",
    "- memmap arrays with num_workers > 0 degrades performance\n",
    "- zarr arrays with num_workers > 0 improves performance\n",
    "\n",
    "Remember that zarr arrays can be read in parallel while memmap arrays can't. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://github.com/timeseriesAI/tsai/blob/main/tutorial_nbs/images/ZarrvsMemmap.png?raw=true\"  width=\"500\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conclusion ✅"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook we have learned how we can train a model with a large dataset in tsai. By using zarr arrays,  setting the chunks parameter to (1,-1,-1) aand num_workers>0, we can really speed up training 2.5x!!\n",
    "\n",
    "Key points: \n",
    "\n",
    "- if you dataset fits in memory, use np.arrays (or np.memmap).\n",
    "- if your dataset doesn't fit in memory, our recommended format is zarr with chunks = (1, None, None)\n",
    "- You should also try setting num_workers>0, as zarr arrays support parallel computing (which means that multiple concurrent read operations may occur). This usually improves performance too. \n",
    "- By using all this you'll be able to significantly reduce your training time when using out-of-memory datasets. \n",
    "- `tsai` currently supports multiple input data types, including: \n",
    "\n",
    "   * np.ndarray\n",
    "   * list\n",
    "   * L\n",
    "   * np.memmap\n",
    "   * dask.array\n",
    "   * xarray\n",
    "   * zarr\n",
    "\n",
    "   The last 4 have the benefit that they can be used on-disk (out-of-core learning), with larger-than-memory datasets.\n",
    "\n",
    "We hope you've find this tutorial useful. If you have any questions/ issues don't hesitate to visit [tsai](https://github.com/timeseriesAI/tsai) in GitHub!!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Additional tests 🥴"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We already know that it's faster to index large zarr arrays compared to memmap arrays. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 loop, best of 5: 1.78 s per loop\n"
     ]
    }
   ],
   "source": [
    "%timeit X_large[np.sort(random_choice(len(X_large_zarr), 512, False))]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 loop, best of 5: 1.27 s per loop\n"
     ]
    }
   ],
   "source": [
    "%timeit X_large_zarr.oindex[np.sort(random_choice(len(X_large_zarr), 512, False))]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But another key difference is multiprocessing. \n",
    "\n",
    "Zarr arrays can be read in parallel by multiple workers, while memmap arrays can't. This means that when we increase num_workers with memmap arrays we usually see a drop in performance. However, performance improves considerably when using num_workers > 0 with zarr arrays."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def cycle_dl_estimate(dl, iters=10):\n",
    "    iters = min(iters, len(dl))\n",
    "    iterator = iter(dl)\n",
    "    timer.start(False)\n",
    "    for _ in range(iters): next(iterator)\n",
    "    t = timer.stop()\n",
    "    return (t/iters * len(dl)).total_seconds()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tfms = [None, TSClassification()]\n",
    "batch_tfms = TSStandardize(by_sample=True)\n",
    "dls = get_ts_dls(X_large, y_large, splits=splits, tfms=tfms, batch_tfms=batch_tfms, inplace=False, bs=[512, 1_024], num_workers=0)\n",
    "train_dl = dls.train\n",
    "valid_dl = dls.valid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 0 44.2\n",
      " 1 67.7\n",
      " 2 71.4\n",
      " 4 87.7\n",
      " 8 114.7\n"
     ]
    }
   ],
   "source": [
    "for nw in [0, 1, 2, 4, 8]:\n",
    "    train_dl.fake_l.num_workers = nw\n",
    "    print(f'{nw:2} {cycle_dl_estimate(train_dl)/60:.1f}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tfms = [None, TSClassification()]\n",
    "batch_tfms = TSStandardize(by_sample=True)\n",
    "dls = get_ts_dls(X_large_zarr, y_large_zarr, splits=splits, tfms=tfms, batch_tfms=batch_tfms, inplace=False, bs=[512, 1_024], num_workers=0)\n",
    "train_dl = dls.train\n",
    "valid_dl = dls.valid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 0 49.1\n",
      " 1 35.0\n",
      " 2 26.4\n",
      " 4 30.6\n",
      " 8 40.4\n"
     ]
    }
   ],
   "source": [
    "for nw in [0, 1, 2, 4, 8]:\n",
    "    train_dl.fake_l.num_workers = nw\n",
    "    print(f'{nw:2} {cycle_dl_estimate(train_dl)/60:.1f}')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
